CAPTCHAs can still detect AI agents • Hacker News Board

Re: CAPTCHAs can still detect AI agents

#11

Post by dpoloncsak » Fri, May 29, 2026, 5:45 PM UTC

catsrus wrote: post #4

BiteCode_dev wrote: post #2

Until they learn to do that. So cat and mouse. So nothing new.

think the point is that they can't just "learn to do that", because to do so would mean solving human mind (that famously hasn't been going well)

until Google trains an AI model off that data, too

Re: CAPTCHAs can still detect AI agents

#12

Post by Cider9986 » Fri, May 29, 2026, 5:46 PM UTC

CAPTCHAs are great. Exploiters get around them with proprietary anti-detect browsers and unethical residential proxies, while privacy browsers and affordable privacy VPNs get blocked and shadowbanned to death.

Fingerprint.com, while not a CAPTCHA, gives you +3 suspicious score just for using privacy settings like adblock on your browser. This makes it harder to sign up for any sites that use fingerprint.com.

https://github.com/CloakHQ/CloakBrowser is a good anti-detect browser as well as CAPTCHA bypass which is honestly fun to use coming from privacy browsers because every site just works and captchas get solved.

Re: CAPTCHAs can still detect AI agents

#13

Post by amirhirsch » Fri, May 29, 2026, 5:57 PM UTC

cute_boi wrote: post #3

I’ve been using Claude Opus 4.7 with Chrome MCP, and it has worked successfully about 95% of the time. However, I’ve failed various hCaptcha challenges.

The thing many people miss is that the challenge itself isn't the primary signal. The challenge creates an opportunity to observe user activity. You're browser is also fingerprinted.

Re: CAPTCHAs can still detect AI agents

#14

Post by timshell » Fri, May 29, 2026, 5:57 PM UTC

docheinestages wrote: post #6

I think it's just a game of cat and mouse. It might be easier to catch naive AI agents that are not fine-tuned for specific CAPTCHA tasks with human behavior, can't recognize new challenges, don't know when to stop and ask a human, and just want to brute force their way with limited or no specialized harness and tools available.

This is relatively close to our conclusion from the paper: unless agents are specifically trained for the task and know all the information ahead of time, they're not able to generalize from one cognitive CAPTCHA to another

Re: CAPTCHAs can still detect AI agents

#15

Post by timshell » Fri, May 29, 2026, 5:58 PM UTC

technotarek wrote: post #8

Apparently CloudFlare’s turnstile can’t, as evidenced by several public-facing CRUD and mail routines we maintain that no longer are warding off the spam.

Yeah, we benchmarked against a few bot detection provides end of last year (https://research.roundtable.ai/bot-benchmarking/), and Turnstile didn't do great when it came to AI agent detection. We hypothesized that Turnstile primarily focuses on device/network characteristics, which AI agents can bypass

Re: CAPTCHAs can still detect AI agents

#16

Post by timshell » Fri, May 29, 2026, 6:00 PM UTC

dpoloncsak wrote: post #11

catsrus wrote: post #4

Earlier quoted context omitted.

think the point is that they can't just "learn to do that", because to do so would mean solving human mind (that famously hasn't been going well)

until Google trains an AI model off that data, too

I mean, their CAPTCHAs presumably have tons of data collected over the years, and they can't detect a pretty clear AI agent here: https://www.youtube.com/watch?v=UeTpCdUc4Ls

Re: CAPTCHAs can still detect AI agents

#17

Post by kjok » Fri, May 29, 2026, 6:05 PM UTC

Adversaries do not have to wait for LLM models to evolve to mimic human process, they can simply evade the detection JavaScript that evaluates similarity. JavaScript is visible, can easily be reverse-engineered.

Re: CAPTCHAs can still detect AI agents

#18

Post by wonkyfruit » Fri, May 29, 2026, 6:22 PM UTC

I had to do a Captcha the other day, and the letters looked awful, so I clicked the speaker for an audible Captcha instead. I was even more horrified. The sound was almost painful. Sharp noise blasting as a high pitched tinny voice bellowed numbers at me. I honestly don't know how blind people use the internet these days with such blockers in place, and that's kind of sad. The cookie banners, the captchas and the bots and laws that made both appear have kinda en$hittified humanity's greatest communication tool.

Re: CAPTCHAs can still detect AI agents

#19

Post by xracy » Fri, May 29, 2026, 6:24 PM UTC

This feels like the kind of thing where, "you must be at least this human to pass" and that it just otherwise mostly wastes your time if you're a robot would cover most of what Captchas are useful for.

Like, if it takes you 3-5 seconds to get through a captcha as a human, as long as every single event has that effort added, the impact to something trying to use/reuse the end-page is way worse if you're a robot than if you're a human.

I can see a few usecases where it would still be valuable to continue the game of cat-and-mouse, but I feel like solving for consistency of human experience of your website, may actually be more punishing to anything trying to bypass it.

Re: CAPTCHAs can still detect AI agents

#20

Post by yrds96 » Fri, May 29, 2026, 6:28 PM UTC

- LLMs can't learn, therefore, LLMs are only good for things on which they are trained. - Captchas are not friendly with trial and error, so agentic solutions also don't help. - It's impractical to train LLMs on everything. - We humans are capable of creating infinite ways of captchas.

While each of these sentences is true, captchas will always win against LLMs.