Can we just get rid of them now, they are so annoying and basically useless.
Benchmarking leading AI agents against Google reCAPTCHA v2
51–60 of 104 posts
Re: Benchmarking leading AI agents against Google reCAPTCHA v2
#52I’m sure they do better than me. Sometimes I get stuck on an endless loop of buses and fire hydrants. Also, when they ask you to identify traffic lights, do you select the post? And when it’s motor/bycicles, do you select the guy riding it?
Pro tip, select a section you know is wrong, then de select it before submitting. Seems to help prove you are not a bot.
Re: Benchmarking leading AI agents against Google reCAPTCHA v2
#53Earlier quoted context omitted.
The solvers are a problem but they give themselves away when they incorrectly fake devices or run out of context. I run a bot detection SaaS and we've had some success blocking them. Their advertised solve times are also wildly inaccurate. They take ages to return a successful token, if at all. The number of companies providing bot mitigation is also growing rapidly, making it difficult for the solvers to stay on top…
> when they incorrectly fake devices And how often does this happen? Do you have any proof? Most YC companies building browser agents have built-in captcha solvers.
Re: Benchmarking leading AI agents against Google reCAPTCHA v2
#54Wow. Cross-tile performance was 0-2%. That's the challenge where you select all of the tiles containing an item where the single item is in a subset of tiles. As opposed to all the tiles that contain the item type (static - 60% max) and the reload version (21% max). Seems to really highlight how far these things are from reasoning or human level intelligence. Although to be fair, the cross-tile is the one I perform w…
I think the prompt is probably at fault here. You can use LLMs for object segmentation and they do fairly well, less than 1% seems too low.
Re: Benchmarking leading AI agents against Google reCAPTCHA v2
#55I’m sure they do better than me. Sometimes I get stuck on an endless loop of buses and fire hydrants. Also, when they ask you to identify traffic lights, do you select the post? And when it’s motor/bycicles, do you select the guy riding it?
Testing those same captcha on Google Chrome improved my accuracy by at least an order of magnitude. Either that or it was never about the buses and fire hydrants.
It doesn't catch OpenAI even though the mouse/click behavior is clearly pretty botlike. One hypothesis is that Google reCAPTCHA is overindexing on browser patterns rather than behavioral movement
Re: Benchmarking leading AI agents against Google reCAPTCHA v2
#56Earlier quoted context omitted.
It's a known "issue" of reCaptcha, and many other systems like it. If it thinks you're a bot, it will "fail" the first few correct solves before it lets you through. The worst offenders will just loop you forever, no matter how many solves you get right.
stock Chrome logged into a Google account = definitely not a bot. here, click a few fire hydrants and come on in :^) I sincerely wish all the folx at Google directly responsible for this particular user acquisition strategy to get every cancer available in California.
Re: Benchmarking leading AI agents against Google reCAPTCHA v2
#57I’m sure they do better than me. Sometimes I get stuck on an endless loop of buses and fire hydrants. Also, when they ask you to identify traffic lights, do you select the post? And when it’s motor/bycicles, do you select the guy riding it?
Re: Benchmarking leading AI agents against Google reCAPTCHA v2
#58Re: Benchmarking leading AI agents against Google reCAPTCHA v2
#59I’m sure they do better than me. Sometimes I get stuck on an endless loop of buses and fire hydrants. Also, when they ask you to identify traffic lights, do you select the post? And when it’s motor/bycicles, do you select the guy riding it?
The buses and fire hydrants are easy. It is the bicycles. If it goes a pixel over the next box, do I select the next box? Is the pole part of the traffic light? And the guy as you say. There is a special place in hell for the inventor of reCaptcha (and for all of Cloudflare staff as fas as I am concerned!)
Re: Benchmarking leading AI agents against Google reCAPTCHA v2
#60Earlier quoted context omitted.
The buses and fire hydrants are easy. It is the bicycles. If it goes a pixel over the next box, do I select the next box? Is the pole part of the traffic light? And the guy as you say. There is a special place in hell for the inventor of reCaptcha (and for all of Cloudflare staff as fas as I am concerned!)
It doesn't matter. Select it of you think other people would select it too.