An AI CAPTCHA solver talked itself out of the right answer
You have probably spent a few seconds of your life turning a picture until it lines up. Some sites, instead of asking you to tick a box, show you a circular chunk of a photo that has been spun around, and you drag it until the inside matches the ring around it. Simple enough. Annoying enough.
Two researchers at Bern University of Applied Sciences wrote a script that solves one of those in 0.006 seconds. It uses circle-detection math from the 1970s and a signal-matching technique that predates everything currently being called AI. It got all ten test puzzles right, every time, faster than you can blink.
Examples of the three CAPTCHA types considered: open-circle, rotation, and insects selection (Source: Research paper)
Then they asked the expensive AI models to do it. Gemini 3.1 Pro took 67 seconds and got seven out of ten. GPT-4o and Grok each got one. There are only eight possible answers, so flipping a coin badly would have done better.
None of that is the interesting part.
The model decided it knew better
Next they let the AI call that fast script as a helper. The model gets the correct answer handed to it and passes it along. A pocket calculator, basically, with instructions to stop doing arithmetic in its head.
GPT-4o and Grok took the answer and scored perfectly. Gemini took the answer and started arguing.
The puzzles came off real websites, so a few of them are slightly crooked to begin with. One lined up best at around 129 degrees, but the site only accepts answers in 45-degree steps, so the script proposed 135 and was right to. Gemini looked at that, decided it still seemed a bit off, and overruled the tool. Doing that cost it a fifth of its score.
Its eyes were fine. Its instinct to overrule a machine that was right was the problem. If you are building anything where a language model supervises a tool that already works, that is the part worth remembering, because the usual sales pitch is that the model catches errors.
They can see it, they just cannot say where it is
The same strangeness runs through the rest of the study.
One puzzle fills the screen with colored rings and asks you to click the one with a gap in it. The models mostly failed. But the researchers made them explain their thinking, and the explanations were right: one described the target as a cyan ring in the lower central area, which it was. It could see the thing. It just could not produce a coordinate accurate enough to click it.
The bug grid is better. Nine squares, pick the ones with insects. Grok found the insects, then called the middle-left square number 5 when it was number 4, and at one point answered square 10 in a grid that has nine. Switching on its reasoning mode, without changing anything about what it was looking at, took it from near-useless to nearly perfect.
The privacy angle
There is one more thing sitting in this paper that the authors walk right past.
These particular puzzles exist because the sites hosting them cannot use JavaScript. They cannot use it because their visitors switch it off, and their visitors switch it off because JavaScript is how you get fingerprinted and tracked across the web. No JavaScript means no watching how the mouse moves, no timing the clicks, none of the invisible behavioral signals that mainstream CAPTCHAs quietly rely on to tell humans from bots.
Strip all of that away and what is left is a shape-matching problem. Computer vision solved shape-matching problems decades ago, on hardware that would struggle to run a modern phone app. The thing protecting these sites is a geometry exercise that a laptop can finish 160 times per second.
