Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Google's Street View computer vision can beat reCAPTCHA with 99% accuracy

googleonlinesecurity.blogspot.com

Post reply on HN
Print view View original

91–100 of 159 posts

Re: Google's Street View computer vision can beat reCAPTCHA with 99% accuracy

#91

Post by polshaw »

daenz wrote: post #76
zwegner wrote: post #45

This particular issue (AI performance on captchas) is really quite fascinating. It's an arms race, but the problem is, only one side can win. Google is claiming they have improved their system in some (understandably) unspecified way, but there's only so far this can go. Captchas need to detect whether someone is human, but it has to work for everyone, ideally, even those with disabilities. Any simple task a human ca…

> It's an arms race, but the problem is, only one side can win. I don't disagree, but although this specific problem can be automated, the arms race will continue as newer problems (requiring uniquely human intelligence) are presented. We can keep moving the bar until the singularity. For example, a captcha: "Pick the most unsafe environment", followed by pictures of a house, a park, a volcano, a bed. To solve this,…

While probably not 99% reliability, I think google software could often solve this one too: Reverse image search could provide an image description, which combined with number of search queries for eg 'unsafe $image_description' to give a ranking. Google can already mostly handle synonyms; it does not need a mastery of the area as it can just use correlations and usage patterns.

Also, being solvable for humans and not computers is not the only criteria for a captcha. For example it must be quick and relatively easy for humans, and provide a definitive consistent answer. The brilliance of recaptcha is that it was making the work that humans excelled at actually useful beyond just 'proof of humanity'.

Re: Google's Street View computer vision can beat reCAPTCHA with 99% accuracy

#93

Post by im3w1l »

daenz wrote: post #76
zwegner wrote: post #45

This particular issue (AI performance on captchas) is really quite fascinating. It's an arms race, but the problem is, only one side can win. Google is claiming they have improved their system in some (understandably) unspecified way, but there's only so far this can go. Captchas need to detect whether someone is human, but it has to work for everyone, ideally, even those with disabilities. Any simple task a human ca…

> It's an arms race, but the problem is, only one side can win. I don't disagree, but although this specific problem can be automated, the arms race will continue as newer problems (requiring uniquely human intelligence) are presented. We can keep moving the bar until the singularity. For example, a captcha: "Pick the most unsafe environment", followed by pictures of a house, a park, a volcano, a bed. To solve this,…

>To solve this, AI would have to have a mastery of language, a mastery of object identification, and a mastery of the metadata of those objects as they apply to human safety.

Actually for a captcha to work, you have to have a problem that is easy for an AI to pose and verify the solution to, but hard for AI to solve. This could potentially rule out some classes of working turing tests.

Re: Google's Street View computer vision can beat reCAPTCHA with 99% accuracy

#94

Post by im3w1l »

nileshtrivedi wrote: post #56
zwegner wrote: post #45

This particular issue (AI performance on captchas) is really quite fascinating. It's an arms race, but the problem is, only one side can win. Google is claiming they have improved their system in some (understandably) unspecified way, but there's only so far this can go. Captchas need to detect whether someone is human, but it has to work for everyone, ideally, even those with disabilities. Any simple task a human ca…

Humans have had an easy time for a while. Good to finally see some competition. :) On the other hand, why is that we insist on humanity? None of the systems can block a captcha-solving team based in Nigeria. What we need to block is bad behavior by users and not their humanity.

I think a big problem is Sybil attacks. Let's suppose I make a bot that posts thousands of reports of my computer crashing after using product X. To determine if such a post is bad behavior or not, you have to know if the claim is genuine. That can't be determined by the content of the post itself.

Re: Google's Street View computer vision can beat reCAPTCHA with 99% accuracy

#95

Post by allochthon »

daenz wrote: post #76
zwegner wrote: post #45

This particular issue (AI performance on captchas) is really quite fascinating. It's an arms race, but the problem is, only one side can win. Google is claiming they have improved their system in some (understandably) unspecified way, but there's only so far this can go. Captchas need to detect whether someone is human, but it has to work for everyone, ideally, even those with disabilities. Any simple task a human ca…

> It's an arms race, but the problem is, only one side can win. I don't disagree, but although this specific problem can be automated, the arms race will continue as newer problems (requiring uniquely human intelligence) are presented. We can keep moving the bar until the singularity. For example, a captcha: "Pick the most unsafe environment", followed by pictures of a house, a park, a volcano, a bed. To solve this,…

"Pick the most unsafe environment", followed by pictures of a house, a park, a volcano, a bed.

My sense is that IBM's Watson could tackle this particular question. Also interesting to note that the question introduces external considerations, such as whether or not there's someone dangerous near the house, bed, park, etc. A human respondent will have to ignore the matter of whether there's additional context and second-guess the captcha author to get the problem right -- presumably the most unsafe environment is the volcano.

Re: Google's Street View computer vision can beat reCAPTCHA with 99% accuracy

#96

Post by roywiggins »

bentcorner wrote: post #71
zwegner wrote: post #45

This particular issue (AI performance on captchas) is really quite fascinating. It's an arms race, but the problem is, only one side can win. Google is claiming they have improved their system in some (understandably) unspecified way, but there's only so far this can go. Captchas need to detect whether someone is human, but it has to work for everyone, ideally, even those with disabilities. Any simple task a human ca…

Ask for x BTC on signup, on the promise that it is returned in y weeks. I was going to propose that it's only returned on good behavior, but that gives sites an incentives to harshly police their new user population. Ideally you'd have some sort of service that both parties trust that is holding the BTC.

MetaFilter is a web forum that charges $5 to create an account. Between that and aggressive but professional moderation, spam basically doesn't happen since a couple of obvious spam comments and you're permabanned.

However, it limits the user population to people with a debit card and $5 to spend. A bitcoin-based solution right now would be even more restrictive, since apart from buying bitcoin in person, you pretty much need a bank account to acquire them.

Re: Google's Street View computer vision can beat reCAPTCHA with 99% accuracy

#97

Post by magicalist »

id wrote: post #15
josho wrote: post #10

Interestingly I activated a new gmail account today and during the signup process I experienced the obligatory captcha. It was in two parts, the first looked strikingly like a street view picture of a house number, while the second looked like a traditional captcha. I suspect that google has been using techniques like this to validate their computer vision conclusions. Which makes their 99% assertion even more intere…

It has been like this for a while now. AFAIK you have either - the actual captcha and a word from a book (Google Books) - the actual captcha and a house number from Street View (Google Maps) Only the actual captcha has to be typed in correctly.

All this speculation on whether or not you can type anything for the other word when you can just try it.

http://www.google.com/recaptcha/learnmore

Only worked once for me, and only when I typed a word that was of similar length to the original, but I was only able to get it to work that one time. Leaving it blank or with a period or only four letters would be an extremely obvious thing to check and deny access to.

Re: Google's Street View computer vision can beat reCAPTCHA with 99% accuracy

#98

Post by seryoiupfurds »

id wrote: post #77
akgerber wrote: post #70

Earlier quoted context omitted.

I did when Recaptcha came out and it was consistently obvious which was the unknown part. It's no longer so obvious, in my experience. I believe ReCaptcha presents the same unknown value to several users, and rejects outliers.

In my experience, it's still extremely obvious. It's always the word that's less readable (and doesn't look like it came from a book). And if you have numbers, it's even easier. Recently they added special captchas for untrusted users. If they think you might be a bot, you'll get much harder captchas. http://googleonlinesecurity.blogspot.com/2013/10/recaptcha-j...

I always refresh until they give me a ``hard'' captcha because the numbers, although easier to read, are harder to type.

Re: Google's Street View computer vision can beat reCAPTCHA with 99% accuracy

#100

Post by SilasX »

daenz wrote: post #76
zwegner wrote: post #45

This particular issue (AI performance on captchas) is really quite fascinating. It's an arms race, but the problem is, only one side can win. Google is claiming they have improved their system in some (understandably) unspecified way, but there's only so far this can go. Captchas need to detect whether someone is human, but it has to work for everyone, ideally, even those with disabilities. Any simple task a human ca…

> It's an arms race, but the problem is, only one side can win. I don't disagree, but although this specific problem can be automated, the arms race will continue as newer problems (requiring uniquely human intelligence) are presented. We can keep moving the bar until the singularity. For example, a captcha: "Pick the most unsafe environment", followed by pictures of a house, a park, a volcano, a bed. To solve this,…

But one requirement of a captcha system is that it be easy for a computer to generate. The system you describe seems to require that humans build up a list of things with obvious-to-human answers, which doesn't scale, especially when they have to be designed under the assumption that "the enemy knows the system" (i.e. a worker on some "library of common knowledge" project could defect).

But if you could have an AI generate those problems, it would be capable of solving them too!

Current captcha systems exploit the one-way nature of problems, mainly "character distortion". This is what permits easy computer-generation but not computer-solution, and does not seem to be present in the class of problems you describe: it's just as easy to go from "noun -> adjective" as "adjective -> noun".

Post reply on HN