Dazed and Confused: A Large-Scale Real-World User Study of ReCAPTCHAv (2023)

Woof, some super high quality "research" going on here. Their basis for this claim is that they found a paper from 2016 where the authors did some cookie aging, found it worked and passed that on to Google, who then fixed it. The authors cite this "blatant vulnerability" but then seem to assume that ReCAPTCHA being versioned means that v2 has never changed since it launched? Then they discover the amazing fact that CAPTCHAs require energy and disappear down a conspiracy theory rabbit hole before concluding the entire product - that people pay money for - is actually useless.

The reality is that academics can't contribute much directly to modern anti-spam work and that has been true for a long time. They aren't genuinely spammers so tend to act in ways that don't trigger anti-spam systems (which is correct behavior). The possibility that their simulations of bad activity aren't accurate enough just doesn't occur to them, so then they run around reporting "vulnerabilities". Usually it's easier to tweak the system to make them happy than have them go to the press and cause a scene, so they think they're doing useful work, but it's actually just distracting employees from the actual job of spam fighting. Back in 2020 Twitter publicly lost patience with this type of university output and slammed it as "extremely limited" and "behind the curve" [1]. Citing an eight year old paper as if it's still relevant is a classic example of this problem. The publish-and-cite model is designed for investigating natural laws, not black box commercial systems that can change on a weekly basis or faster. Their supervisor should really have stopped them citing something that old.

The other problem is that because researchers don't understand spammers very well they tend to assume that anything they can do, spammers can also do. In reality spam is purely a profit/loss optimization problem. Spammers are business people who want to make money. The goal is to throw just enough speedbumps in their path to make their profit margin negative, at which point they give up and the spam goes away, without also tipping your own numbers into the red by over-spending or hurting the UX too badly. Making things tougher than necessary risks pushing your own margins negative, making things easier than necessary risks users getting annoyed at the spam and leaving. The goal is to ride the edge of the wave as closely as possible. When spammers find a trick that works they keep it as a trade secret and sell the results of using the trick, not the trick itself, and so disrupting the relationships between black market suppliers is an important tactic. CAPTCHAs in particular are just a throttle intended to slow down low grade spam; if you're actually spending 20-30 seconds automatically solving image puzzles with an 80% success rate you haven't beaten anything, because slowing you down that much was the goal in the first place.

This environment is nothing like academia, where people are getting funded by the government to spend months doing work basically for the heck of it. In academia it's reasonable to spend a year gathering data, running experiments, training ML models, solve a tiny subset of the actual problem spammers face, ignore the cost of doing all that because it's government subsidized, and then announce you've defeated something. If they were really spammers they'd given up and moved on to a different scheme long before that (in a surprising number of cases, this new scheme will be getting a proper job). So whether they realize it or not, all this line of work really does is funnel tax money into the spam ecosystem - exactly what we don't want.

Source: worked in web anti-spam for a while.

[1] https://blog.x.com/en_us/topics/company/2020/bot-or-not "the threat has evolved and the narrative on what’s actually going on is increasingly behind the curve."

Just load a captcha-blocked page in a fresh browser with no google accounts every being used - you got like a 1% chance of getting through. Login to google, and BOOM you're in.