Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Benchmarking leading AI agents against Google reCAPTCHA v2

research.roundtable.ai

Post reply on HN
Print view View original

71–80 of 104 posts

Re: Benchmarking leading AI agents against Google reCAPTCHA v2

#71

Post by max51 »

terminalshort wrote: post #67
Xenoamorphous wrote: post #4

I’m sure they do better than me. Sometimes I get stuck on an endless loop of buses and fire hydrants. Also, when they ask you to identify traffic lights, do you select the post? And when it’s motor/bycicles, do you select the guy riding it?

There is a doom loop mode where it doesn't matter how many you solve or even if you get them correct. My source for this works on this product at Google.

That doesn't surprise me. I find it hard to believe it's a pure coincidence that I would get stuck in the loop regularly when I'm on the university wifi but it would never happen anywhere else ever. After a dozen try, I would remote connect to my home pc and it would magically work on the first try every single time.

Re: Benchmarking leading AI agents against Google reCAPTCHA v2

#72

Post by perfmode »

Xenoamorphous wrote: post #4

I’m sure they do better than me. Sometimes I get stuck on an endless loop of buses and fire hydrants. Also, when they ask you to identify traffic lights, do you select the post? And when it’s motor/bycicles, do you select the guy riding it?

> do you select the guy riding it? do you select the post?

Just select as _you_ would. As _you_ do.

Imperfection and differing judgments are inherent to being human. The CAPTCHA also measures your mouse movement on the X and Y axes and the timing of your clicks.

Re: Benchmarking leading AI agents against Google reCAPTCHA v2

#73

Post by dfxm12 »

cm2187 wrote: post #57
Xenoamorphous wrote: post #4

I’m sure they do better than me. Sometimes I get stuck on an endless loop of buses and fire hydrants. Also, when they ask you to identify traffic lights, do you select the post? And when it’s motor/bycicles, do you select the guy riding it?

The buses and fire hydrants are easy. It is the bicycles. If it goes a pixel over the next box, do I select the next box? Is the pole part of the traffic light? And the guy as you say. There is a special place in hell for the inventor of reCaptcha (and for all of Cloudflare staff as fas as I am concerned!)

There's not a right or wrong answer. They're just building consensus for self driving vehicles.

Re: Benchmarking leading AI agents against Google reCAPTCHA v2

#74

Post by Tostino »

michaelt wrote: post #68
varenc wrote: post #56

Earlier quoted context omitted.

I would think that when you're viewing recaptcha on a site, if you have 3rd party cookies disabled the embedded recaptcha script won't have anyway of connecting you with your Google account, even if you're logged in. At least that's how disabling 3rd party cookies is supposed to work.

Of course, if you have 3rd party cookies disabled, Google would never link your recaptcha activity to your Google account. They just link it to your IP address, browser, operating system, screen resolution, set of fonts, plugins, timezone, mouse movements, GPU, number of CPU cores, and of course the fact you've got third party cookies disabled.

"Oh, that's interesting...there is one other user that matches all of that metadata"

Re: Benchmarking leading AI agents against Google reCAPTCHA v2

#75

Post by [deleted] »

perfmode wrote: post #72
Xenoamorphous wrote: post #4

I’m sure they do better than me. Sometimes I get stuck on an endless loop of buses and fire hydrants. Also, when they ask you to identify traffic lights, do you select the post? And when it’s motor/bycicles, do you select the guy riding it?

> do you select the guy riding it? do you select the post? Just select as _you_ would. As _you_ do. Imperfection and differing judgments are inherent to being human. The CAPTCHA also measures your mouse movement on the X and Y axes and the timing of your clicks.

[deleted]

Re: Benchmarking leading AI agents against Google reCAPTCHA v2

#76

Post by jorvi »

cm2187 wrote: post #60
charcircuit wrote: post #59

Earlier quoted context omitted.

It doesn't matter. Select it of you think other people would select it too.

That's the thing, you could go either way. I am not sure I can answer the question "what would a resonable person click?".

The trick is to pretend you're an idiot. If the bicycle and the person on it map mostly to a rectangle of 8 squares, most people will be so stupid or hasty that they'll click that, nevermind that a human is not part of the bicycle.

The same is true with, say, buses. See an image of a delivery van? Bus! It asks you select all cars and you see no car but a vague pixel blob that someone stupid would identify as a car? Car!

One of the few things that this doesn't work with is stairs, because the side of stairs being stairs or not is something apparently no one can agree on.

Re: Benchmarking leading AI agents against Google reCAPTCHA v2

#77

Post by nistiminic »

Xenoamorphous wrote: post #4

I’m sure they do better than me. Sometimes I get stuck on an endless loop of buses and fire hydrants. Also, when they ask you to identify traffic lights, do you select the post? And when it’s motor/bycicles, do you select the guy riding it?

Just select the audio option. It's faster and easier. Maybe it's because google doesn't care about training on speech to text. I usually write something random for one word and get the other word correct. I can even write "bzzzzt" at the beginning. They don't care because they aren't focused on training on that data.

Now I think of it, it's really a failure that AI didn't use this and went with guessing which square of an image to select.

Re: Benchmarking leading AI agents against Google reCAPTCHA v2

#79

Post by jgalt212 »

terminalshort wrote: post #67
Xenoamorphous wrote: post #4

I’m sure they do better than me. Sometimes I get stuck on an endless loop of buses and fire hydrants. Also, when they ask you to identify traffic lights, do you select the post? And when it’s motor/bycicles, do you select the guy riding it?

There is a doom loop mode where it doesn't matter how many you solve or even if you get them correct. My source for this works on this product at Google.

Firefox and Ubuntu can reliably trigger this mode.

Re: Benchmarking leading AI agents against Google reCAPTCHA v2

#80

Post by [deleted] »

cubefox wrote: post #27
ajsnigrutin wrote: post #22

So, when do we reach a level where AI is better than humans and we remove captcha from pages alltogether? If you don't want bots to read content, don't put it online, you're just inconveniencing real people now.

They can also sign up and post spam/scams. There are a lot of those spam bots on YouTube, and there probably would be a lot more without any bot protection. Another issue is aggressive scrapers effectively DOSing a website. Some defense against bots is necessary.

[deleted]
Post reply on HN