Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Images that fool computer vision raise security concerns

news.cornell.edu

Post reply on HN
Print view View original

41–50 of 220 posts

Re: Images that fool computer vision raise security concerns

#41

Post by platz »

msandford wrote: post #3

This is some excellent research! It reminds me of the CV dazzle anti-facial-recognition makeup that made the rounds a while ago: http://www.theatlantic.com/features/archive/2014/07/makeup/3... This definitely reinforces my belief that having humans in the loop is not only desirable but necessary. For the majority of human history minus a few years you could only be accused of a crime by another human being. I'd like…

Someone talking about the kinect said that most all facial recognition algorithms require the face to be level horizontally; if the face is tilted at an angle then it doesn't work. not sure if true but interesting observation.

Re: Images that fool computer vision raise security concerns

#42

Post by JoeAltmaier »

maxerickson wrote: post #39
JoeAltmaier wrote: post #37

Earlier quoted context omitted.

The OP shows that computers DO NOT have a better understanding. Its evident they have no understanding at all; they are simply doing math on pixels and latching on to coincidental patters of color or shading. People recognize things by building a 3D model in their head, then comparing that to billions of experiential models, finding a match and then using cognition to test that match. "Is that a bird? No, its just a…

How could I have better put So at the limit, it could ? I meant to talk about what some hypothetical future system could do (which I think was a reasonable context given the comment I replied to), not to characterize current systems.

Sorry, I re-read and see that.

To get there, computers will clearly have to change utterly their approach. A cascaded approach of quick-math followed by a more 'cognitive' approach on possible matches, could definitely improve on the current state of affairs.

Re: Images that fool computer vision raise security concerns

#43

Post by compbio »

woodchuck64 wrote: post #38

"“We realized that the neural nets did not encode knowledge necessary to produce an image of a fire truck, only the knowledge necessary to tell fire trucks apart from other classes,” [Yosinski] explained." This seems markedly different from biological neural networks. Is the difference one of network structure/algorithm or rather the fact that biological neural networks (in human image processing) actually have time…

Deep nets are only loosely inspired by neurobiology. That's why LeCun calls them "convolutional nets" and not "convolutional neural nets" and prefers "nodes" over "neurons".

It is, however, possible to have a deep net produce 3D models/images: https://www.youtube.com/watch?v=QCSW4isBDL0 "Learning to Generate Chairs with Convolutional Neural Networks".

I also suspect a different part of cognition is used when humans are asked to recreate a "fire truck" than when humans are asked to classify a "fire truck" from a "car". The former seems closer to using memory ("what did the last five fire trucks I saw look like?"). A fairly recent addition to deep nets is making use of memory: http://arxiv.org/pdf/1410.5401.pdf "Neural Turing Machines". So the difference may quickly become less significant.

Re: Images that fool computer vision raise security concerns

#44

Post by vinceguidry »

mikeash wrote: post #27
toomuchtodo wrote: post #11

Earlier quoted context omitted.

> If everyone is breaking so many laws that the police and courts can't keep up it doesn't mean that humanity is broken. It means that the law has gotten so far out of sync with humanity that the law is broken. People make the laws, not the other way around. The world would be a much better place if more people realized this.

This always frustrates me when discussions of plea bargaining and the right to trial come up, and the argument is given that plea bargaining is a necessity because the courts would be horribly overloaded if every case went to trial. If the system doesn't have the resources to give every accused criminal a fair trial, then either you're making too many criminals, the system doesn't have enough resources, or both. Bypa…

The problem is that you can't get very far into the discussion before all the available options involve overhauling the entire system. And there's no way, realistically, to do that.

Re: Images that fool computer vision raise security concerns

#45

Post by toomuchtodo »

vinceguidry wrote: post #44
mikeash wrote: post #27

Earlier quoted context omitted.

This always frustrates me when discussions of plea bargaining and the right to trial come up, and the argument is given that plea bargaining is a necessity because the courts would be horribly overloaded if every case went to trial. If the system doesn't have the resources to give every accused criminal a fair trial, then either you're making too many criminals, the system doesn't have enough resources, or both. Bypa…

The problem is that you can't get very far into the discussion before all the available options involve overhauling the entire system. And there's no way, realistically, to do that.

> And there's no way, realistically, to do that.

Yes, yes there is. We start with the most victimless crimes (drug possession for personal consumption) and go from there.

Police unions will fight it. Prison unions will fight it. The court system will fight it. Nevertheless, it must be done.

Re: Images that fool computer vision raise security concerns

#46

Post by pavel_lishin »

anon4 wrote: post #19
msandford wrote: post #8

Earlier quoted context omitted.

Welp, it's official. I'm a computer.

No, you're the maintenance guy. Computers don't care about formatting. The correct answer is A, because "A" is drawn with just three straight lines and computers like straight lines.

But it arguably takes more data to encode an A - three lines, which means six endpoints, plus whatever signifies the command "draw a straight line." At minimum, that's seven pieces of data.

A C, however, can be drawn as half of a circle - one command to draw an arc, a center point, a radius, and the start and stop angles. Five pieces of data.

(This is assuming, of course, that computers prefer minimal amounts of data. If that's wrong, then the computer would obviously prefer B. You need more data to describe it.)

Re: Images that fool computer vision raise security concerns

#47

Post by incongruity »

mikeash wrote: post #27
toomuchtodo wrote: post #11

Earlier quoted context omitted.

> If everyone is breaking so many laws that the police and courts can't keep up it doesn't mean that humanity is broken. It means that the law has gotten so far out of sync with humanity that the law is broken. People make the laws, not the other way around. The world would be a much better place if more people realized this.

This always frustrates me when discussions of plea bargaining and the right to trial come up, and the argument is given that plea bargaining is a necessity because the courts would be horribly overloaded if every case went to trial. If the system doesn't have the resources to give every accused criminal a fair trial, then either you're making too many criminals, the system doesn't have enough resources, or both. Bypa…

While I largely agree with you, it strikes me that you've missed an option --

It might be that there's something inefficient in the trial process that holds no weight on "fairness". I'm not a legal expert by any means, so it may not be the case, but it seems as though it's a possibility.

Re: Images that fool computer vision raise security concerns

#48

Post by jgrowl »

compbio wrote: post #16
jahnu wrote: post #2

> But computers don’t process images the way humans do, Yosinski said. This means that come the singularity AIs will have to use AI specific CAPTCHAs in order to distinguish between humans (aided by dumb computers) and other AIs.

If we have a singularity AI then we don't need CAPTCHAs: For all intents and purposes an AI is equal or superior to humans, there is no distinction to be made. The Turing Test is a form of CAPTCHA. Also, one could make CAPTCHA's incredibly hard. Humans and dumb computers won't be able to solve it, and singular AI's get a pass. So give access to anyone who isn't able to solve the CAPTCHA and redirect the singularity A…

It makes me chuckle thinking about machine intelligence being able to solve a captcha when humans can't.

I get a mental image of a future where computers try to keep us pesky humans out of their websites.

Re: Images that fool computer vision raise security concerns

#49

Post by [deleted] »

userbinator wrote: post #14
compbio wrote: post #10

It is good to know that they need access to a lot of predictions from a net, before they can create an image that will "fool" the net, but look alien to humans. Secondly, this doesn't account for ensembling: "fool me once, shame on you. Fool me twice...". Since the images are crafted for a single net, a majority vote should not be fooled by these images. I suspect this effect rapidly goes away when adding more nets (…

Furthermore, I am seeing the security concerns, but I figure this is far from a practical attack Perhaps it's a sign of the times that almost every discovery that could possibly be related to security in some way, does. I have a feeling that if this was a decade or two ago, the sentiment would be very different. ("Can you figure out what a computer thinks these images are?") Also, the image labeled "baseball" immedia…

[deleted]

Re: Images that fool computer vision raise security concerns

#50

Post by JoeAltmaier »

jgrowl wrote: post #48
compbio wrote: post #16

Earlier quoted context omitted.

If we have a singularity AI then we don't need CAPTCHAs: For all intents and purposes an AI is equal or superior to humans, there is no distinction to be made. The Turing Test is a form of CAPTCHA. Also, one could make CAPTCHA's incredibly hard. Humans and dumb computers won't be able to solve it, and singular AI's get a pass. So give access to anyone who isn't able to solve the CAPTCHA and redirect the singularity A…

It makes me chuckle thinking about machine intelligence being able to solve a captcha when humans can't. I get a mental image of a future where computers try to keep us pesky humans out of their websites.

Cool! And it would be so easy. Just show one of those noisy thumbnails, and see if the human can identify it as a 'panda'
Post reply on HN