Maybe this is the algorithm making art
Images that fool computer vision raise security concerns
61–70 of 220 posts
Re: Images that fool computer vision raise security concerns
#62They can only fool the DNN because they know their weights. In theory, you could do the same for people's brains. As long as you don't publish the specs of your net you should be fine I guess.
That's a good point! But your proposed solution is security by obscurity. What if a rogue employee trained a backdoor into your DNN? What if the NSA asked vendors for a master key?
The purpose of this research is to work as a proof of concept. Sure, this iteration needs to cheat slightly to achieve its results. However, looking at the images that aren't just noise, it seems to be possible to construct less specialized images that fool many nets.
Re: Images that fool computer vision raise security concerns
#63Earlier quoted context omitted.
This always frustrates me when discussions of plea bargaining and the right to trial come up, and the argument is given that plea bargaining is a necessity because the courts would be horribly overloaded if every case went to trial. If the system doesn't have the resources to give every accused criminal a fair trial, then either you're making too many criminals, the system doesn't have enough resources, or both. Bypa…
> you're making too many criminals 80% of property crime goes unsolved, so unless you legalize vandalism and theft there are going to be criminals.
Or fix the underlying problems that causes vandalism and theft.
Re: Images that fool computer vision raise security concerns
#64This raises an interesting question. Do we want computers to see "correctly", or to see how we see? Would a preferred computer vision system experience the Checker shadow illusion? http://en.wikipedia.org/wiki/Checker_shadow_illusion If yes, computer vision will be as fallible as ours. If no, then there will always be examples, like the ones presented, where computers will see something different than humans.
The question is, what do you mean by "correctly"?
When we look at the Checker Shadow Illusion, our brains are automatically "parsing" that image into a 3D scene and compensating for the lighting and shadowing. The reason you see square B as being lighter than A is because, if you could reach into the image and remove the cylinder so it's not casting a shadow anymore, square B would be lighter than A. Our brain doesn't think about color in terms of absolute hex values. Instead it tries to compensate for the lighting and positioning of elements, assuming that they're similar to what we would see in real life—a challenge which, by the way, computer vision systems have always struggled with.
That sounds pretty "correct" to me.
Re: Images that fool computer vision raise security concerns
#65This raises an interesting question. Do we want computers to see "correctly", or to see how we see? Would a preferred computer vision system experience the Checker shadow illusion? http://en.wikipedia.org/wiki/Checker_shadow_illusion If yes, computer vision will be as fallible as ours. If no, then there will always be examples, like the ones presented, where computers will see something different than humans.
[deleted]
Re: Images that fool computer vision raise security concerns
#66Earlier quoted context omitted.
It's not excellent research. All they did is create a naive NN without any feature recognition and noted that it has a high rate of false positives when presented with random imagery. Duh.
Really? From the article: "They tested with two widely used DNN systems that have been trained on massive image databases." Is this wrong?
GIGO. Just because they used a "Deep" architecture doesn't mean it was suitable for the task.
Re: Images that fool computer vision raise security concerns
#67Earlier quoted context omitted.
This always frustrates me when discussions of plea bargaining and the right to trial come up, and the argument is given that plea bargaining is a necessity because the courts would be horribly overloaded if every case went to trial. If the system doesn't have the resources to give every accused criminal a fair trial, then either you're making too many criminals, the system doesn't have enough resources, or both. Bypa…
> you're making too many criminals 80% of property crime goes unsolved, so unless you legalize vandalism and theft there are going to be criminals.
Perhaps this is another way out of our problem? We could encourage police to "solve" property crimes rather than drug crimes? In analogy to their awful drug-war asset seizures, perhaps they could get a cut of items recovered/reimbursed? I'm sure they would eventually twist such an incentive structure into something else that's awful and unconscionable, but at least in the meantime they might stop with the SWAT raids?
Re: Images that fool computer vision raise security concerns
#68Machine pareidolia?
On a serious not couldn't we keep training the same DNN's using these white noise images as negative examples?
Re: Images that fool computer vision raise security concerns
#69Earlier quoted context omitted.
> you're making too many criminals 80% of property crime goes unsolved, so unless you legalize vandalism and theft there are going to be criminals.
Perhaps this is another way out of our problem? We could encourage police to "solve" property crimes rather than drug crimes? In analogy to their awful drug-war asset seizures, perhaps they could get a cut of items recovered/reimbursed? I'm sure they would eventually twist such an incentive structure into something else that's awful and unconscionable, but at least in the meantime they might stop with the SWAT raids?
Could we not just outlaw SWAT raids?
Re: Images that fool computer vision raise security concerns
#70This is some excellent research! It reminds me of the CV dazzle anti-facial-recognition makeup that made the rounds a while ago: http://www.theatlantic.com/features/archive/2014/07/makeup/3... This definitely reinforces my belief that having humans in the loop is not only desirable but necessary. For the majority of human history minus a few years you could only be accused of a crime by another human being. I'd like…
Someone talking about the kinect said that most all facial recognition algorithms require the face to be level horizontally; if the face is tilted at an angle then it doesn't work. not sure if true but interesting observation.
That's really not that interesting. I work as a research scientist at a face recognition company. If you want, you can detect faces at any angle, but sometimes for speed you will only check upright faces.