They can only fool the DNN because they know their weights. In theory, you could do the same for people's brains.
As long as you don't publish the specs of your net you should be fine I guess.
Images that fool computer vision raise security concerns
51–60 of 220 posts
Re: Images that fool computer vision raise security concerns
#52This raises an interesting question. Do we want computers to see "correctly", or to see how we see? Would a preferred computer vision system experience the Checker shadow illusion? http://en.wikipedia.org/wiki/Checker_shadow_illusion If yes, computer vision will be as fallible as ours. If no, then there will always be examples, like the ones presented, where computers will see something different than humans.
You can always fool any system into a paradox. Very grossly speaking, this comes out of Godel's incompleteness theorem; that with any set of laws you can always get P=~P out of the set. How this paradox looks, acts, or feels is interesting and possibly artistic, as the OP shows. If anything, I think there is a bit of beauty, art, and cleverness in that.
That isn't quite right. With a sufficiently powerful formal system, you're forced to either have inconsistency or incompleteness - you're describing a system that is inconsistent. It's usually much better to have consistency and to sacrifice completeness. Then you'll have Ps that are true but unprovable, but at least you won't have P=~P which makes the system rather useless.
Re: Images that fool computer vision raise security concerns
#53This is some excellent research! It reminds me of the CV dazzle anti-facial-recognition makeup that made the rounds a while ago: http://www.theatlantic.com/features/archive/2014/07/makeup/3... This definitely reinforces my belief that having humans in the loop is not only desirable but necessary. For the majority of human history minus a few years you could only be accused of a crime by another human being. I'd like…
It's not excellent research. All they did is create a naive NN without any feature recognition and noted that it has a high rate of false positives when presented with random imagery. Duh.
Re: Images that fool computer vision raise security concerns
#54Earlier quoted context omitted.
> If everyone is breaking so many laws that the police and courts can't keep up it doesn't mean that humanity is broken. It means that the law has gotten so far out of sync with humanity that the law is broken. People make the laws, not the other way around. The world would be a much better place if more people realized this.
This always frustrates me when discussions of plea bargaining and the right to trial come up, and the argument is given that plea bargaining is a necessity because the courts would be horribly overloaded if every case went to trial. If the system doesn't have the resources to give every accused criminal a fair trial, then either you're making too many criminals, the system doesn't have enough resources, or both. Bypa…
> you're making too many criminals
80% of property crime goes unsolved, so unless you legalize vandalism and theft there are going to be criminals.
Re: Images that fool computer vision raise security concerns
#55This is some excellent research! It reminds me of the CV dazzle anti-facial-recognition makeup that made the rounds a while ago: http://www.theatlantic.com/features/archive/2014/07/makeup/3... This definitely reinforces my belief that having humans in the loop is not only desirable but necessary. For the majority of human history minus a few years you could only be accused of a crime by another human being. I'd like…
It's not excellent research. All they did is create a naive NN without any feature recognition and noted that it has a high rate of false positives when presented with random imagery. Duh.
Really? From the article:
"They tested with two widely used DNN systems that have been trained on massive image databases."
Is this wrong?
Re: Images that fool computer vision raise security concerns
#56A paper came out that explains this effect and a method of minimizing it: http://arxiv.org/abs/1412.6572 Basically neural networks and many other machine learning methods are highly linear and continuous. So changing an input just slightly should change the output just slightly. If you change all of the inputs slightly in just the right directions, you can manipulate the output arbitrarily. These images are highly op…
> Basically neural networks and many other machine learning methods are highly linear No they're not! You introduce non-linearities like the sigmoid or tanh to make them highly non-linear.
I was thinking the same thing until I scanned through the paper linked above. While neural networks are indeed non-linear, some NNs can still exhibit what amounts to linearity and suffer from adversarial linear perturbations. Example of linearity in NNs that the authors are considering from the paper:
>The linear view of adversarial examples suggests a fast way of generating them. We hypothesize that neural networks are too linear to resist linear adversarial perturbation. LSTMs (Hochreiter & Schmidhuber, 1997), ReLUs (Jarrett et al., 2009; Glorot et al., 2011), and maxout networks (Goodfellow et al., 2013c) are all intentionally designed to behave in very linear ways, so that they are easier to optimize. More nonlinear models such as sigmoid networks are carefully tuned to spend most of their time in the non-saturating, more linear regime for the same reason. This linear behavior suggests that cheap, analytical perturbations of a linear model should also damage neural networks.
Re: Images that fool computer vision raise security concerns
#57This is some excellent research! It reminds me of the CV dazzle anti-facial-recognition makeup that made the rounds a while ago: http://www.theatlantic.com/features/archive/2014/07/makeup/3... This definitely reinforces my belief that having humans in the loop is not only desirable but necessary. For the majority of human history minus a few years you could only be accused of a crime by another human being. I'd like…
It's not just the recognition front end that is problematic. In context of the issue of mis-recognition, Aphyr's tear down of various backends [1] -- which for all we know are in use by the lettered agencies -- gives me the willies.
Re: Images that fool computer vision raise security concerns
#58They can only fool the DNN because they know their weights. In theory, you could do the same for people's brains. As long as you don't publish the specs of your net you should be fine I guess.
If you have an oracle, you don't need to know the net specs.
An oracle, here, would be any version of the system that you can query against repeatedly without suffering too much of a penalty.
Re: Images that fool computer vision raise security concerns
#59This raises an interesting question. Do we want computers to see "correctly", or to see how we see? Would a preferred computer vision system experience the Checker shadow illusion? http://en.wikipedia.org/wiki/Checker_shadow_illusion If yes, computer vision will be as fallible as ours. If no, then there will always be examples, like the ones presented, where computers will see something different than humans.
You can always fool any system into a paradox. Very grossly speaking, this comes out of Godel's incompleteness theorem; that with any set of laws you can always get P=~P out of the set. How this paradox looks, acts, or feels is interesting and possibly artistic, as the OP shows. If anything, I think there is a bit of beauty, art, and cleverness in that.
[deleted]
Re: Images that fool computer vision raise security concerns
#60Does anyone have higher resolution versions of the images used in the article?