Images that fool computer vision raise security concerns
news.cornell.edu
Images that fool computer vision raise security concerns
1–10 of 220 posts
Re: Images that fool computer vision raise security concerns
#2> But computers don’t process images the way humans do, Yosinski said.
This means that come the singularity AIs will have to use AI specific CAPTCHAs in order to distinguish between humans (aided by dumb computers) and other AIs.
Re: Images that fool computer vision raise security concerns
#3This is some excellent research!
It reminds me of the CV dazzle anti-facial-recognition makeup that made the rounds a while ago: http://www.theatlantic.com/features/archive/2014/07/makeup/3...
This definitely reinforces my belief that having humans in the loop is not only desirable but necessary. For the majority of human history minus a few years you could only be accused of a crime by another human being. I'd like to see the trend of automated "enforcement" reversed and codify into law that you MUST be accused by a human being.
If everyone is breaking so many laws that the police and courts can't keep up it doesn't mean that humanity is broken. It means that the law has gotten so far out of sync with humanity that the law is broken. People make the laws, not the other way around.
Re: Images that fool computer vision raise security concerns
#4> But computers don’t process images the way humans do, Yosinski said. This means that come the singularity AIs will have to use AI specific CAPTCHAs in order to distinguish between humans (aided by dumb computers) and other AIs.
[deleted]
Re: Images that fool computer vision raise security concerns
#5Peter Watts mentioned this potential problem in his Rifters series; one explicit example was a neural net that ran a train and was trained via a series of inputs, one of which was a clock in a train station; one day, the clock broke, and the neural net took some action that ended up killing all the passengers. (I forget the details.)
Which is not to say that we should all fear computers more than humans as a consequence; we do inexplicable things, too.
Re: Images that fool computer vision raise security concerns
#6> But computers don’t process images the way humans do, Yosinski said. This means that come the singularity AIs will have to use AI specific CAPTCHAs in order to distinguish between humans (aided by dumb computers) and other AIs.
Which of the following would you most prefer? A: a puppy, B: a pretty flower from your sweetie, or C: a large properly formatted data file?
Re: Images that fool computer vision raise security concerns
#7This raises an interesting question. Do we want computers to see "correctly", or to see how we see?
Would a preferred computer vision system experience the Checker shadow illusion? http://en.wikipedia.org/wiki/Checker_shadow_illusion
If yes, computer vision will be as fallible as ours. If no, then there will always be examples, like the ones presented, where computers will see something different than humans.
Re: Images that fool computer vision raise security concerns
#8> But computers don’t process images the way humans do, Yosinski said. This means that come the singularity AIs will have to use AI specific CAPTCHAs in order to distinguish between humans (aided by dumb computers) and other AIs.
Which of the following would you most prefer? A: a puppy, B: a pretty flower from your sweetie, or C: a large properly formatted data file?
Welp, it's official. I'm a computer.
Re: Images that fool computer vision raise security concerns
#9A paper came out that explains this effect and a method of minimizing it: http://arxiv.org/abs/1412.6572
Basically neural networks and many other machine learning methods are highly linear and continuous. So changing an input just slightly should change the output just slightly. If you change all of the inputs slightly in just the right directions, you can manipulate the output arbitrarily.
These images are highly optimized for this effect and unlikely to occur by random chance. Adding random noise to images doesn't seem to cause it, because for every pixel changed in the right direction, another is changed in the wrong direction.
The researchers found a quick method of generating these images, and found that training on them improved the net a lot. Not just on the adversarial examples.
Re: Images that fool computer vision raise security concerns
#10It is good to know that they need access to a lot of predictions from a net, before they can create an image that will "fool" the net, but look alien to humans. Secondly, this doesn't account for ensembling: "fool me once, shame on you. Fool me twice...". Since the images are crafted for a single net, a majority vote should not be fooled by these images. I suspect this effect rapidly goes away when adding more nets (which is basically industry-standard practice to increase accuracy).
Furthermore, I am seeing the security concerns, but I figure this is far from a practical attack. Deep Learning Classifiers do not act as gatekeepers: You have not much to gain from a single faulty classification. You won't be granted access to secret information if you happen to look like the CEO.