I recently ran Tuskfish through a couple more sweeps after being offered a free deep scan by CodeRabbit and temporary access to Anthropic's frontier model Fable 5.2. Both found a number of minor issues and bugs, which I am in process of tidying up, and I will probably put out a new release (2.3.6) on the weekend.

The good news is that nothing major was found. But each time the models advance they find new issues, although the bugs are becoming progressively more fringe and edge case. The most recent findings are borderline forensic, I doubt that I would ever have found them myself.

One wonders, where does it stop? Are we doomed to be patching our software every six weeks, forever? Can we ever get to a state where even mildly complex software is truly bug-free? I thought I was done after sweeping with Opus 5. But apparently, no.

One also wonders what happens when the AI labs stop providing entry level subscribers with free trial access to their frontier models. In the era of AI, programmers no longer have a choice: We must defensively screen code bases with the tools people are going to use to break them. If that costs hundreds of dollars per run, freelance and open source developers are going to have a problem.

For a programmer, AI models are magic made real. But to be honest, I think we were probably better off without these things. Perhaps, a lot better off.