Live data from Hacker News

Images that fool computer vision raise security concerns

news.cornell.edu

71–80 of 220 posts

Re: Images that fool computer vision raise security concerns

#71
post #55

Earlier quoted context omitted.

Really? From the article: "They tested with two widely used DNN systems that have been trained on massive image databases." Is this wrong?

GIGO. Just because they used a "Deep" architecture doesn't mean it was suitable for the task.

Does your criticism extend to the whole enterprise of facial recognition, or did this research somehow fail to include the secret sauce that makes it work?

Re: Images that fool computer vision raise security concerns

#72

Not directly related but, I was at a security related convention and overheard some people talking about an image that when occupying <3/4 of a frame will crash any digital camera. (Phone, DSLR, IP Camera) Does anyone know anymore information about this image and effect? I imagine it's a bug in some low level firmware of a common IC for digital photography DSP but, I'm very unfamiliar with digital cameras. It also co…

Flying RPG?

Now seriously, modern cameras have face recognition and ton of other features. It should be possible to cause crash.

Re: Images that fool computer vision raise security concerns

#73

A paper came out that explains this effect and a method of minimizing it: http://arxiv.org/abs/1412.6572 Basically neural networks and many other machine learning methods are highly linear and continuous. So changing an input just slightly should change the output just slightly. If you change all of the inputs slightly in just the right directions, you can manipulate the output arbitrarily. These images are highly op…

> Basically neural networks and many other machine learning methods are highly linear No they're not! You introduce non-linearities like the sigmoid or tanh to make them highly non-linear.

Most image recognition neural networks use Relu activations which is just a linear output unless the input is below zero. Even sigmoids/tanh are most linear in the middle region, and weight penalties are used to keep the weights small so they stay in that region.

But it doesn't really matter what activation function you use. The paper argues that its the linear layers between the nonlinearities that are the problem.

Re: Images that fool computer vision raise security concerns

#74
post #27

Earlier quoted context omitted.

> If everyone is breaking so many laws that the police and courts can't keep up it doesn't mean that humanity is broken. It means that the law has gotten so far out of sync with humanity that the law is broken. People make the laws, not the other way around. The world would be a much better place if more people realized this.

This always frustrates me when discussions of plea bargaining and the right to trial come up, and the argument is given that plea bargaining is a necessity because the courts would be horribly overloaded if every case went to trial. If the system doesn't have the resources to give every accused criminal a fair trial, then either you're making too many criminals, the system doesn't have enough resources, or both. Bypa…

Why should every crime go to trial? If the offender says "Fair cop, guv, you've got me bang to rights", why waste everyone's time proving that (s)he did it?

It's basically the same as setting up your conditions to take advantage of short-circuit evaluation. You don't put the time-consuming and resource-hungry part first.

Were there even the slimmest chance of acquittal, few defendants would utter that phrase without there being either a benefit to owning up, or an extra penalty for not doing so. This is what plea bargaining and TICs are for.

If there's a suspicion of false confession, meaning that the suspect may not be the offender, that should normally be sorted out before court, so that the correct offender is tried for the correct offence (e.g. wasting police time if voluntary, or some kind of intimidation/coercion offence if not)?

Re: Images that fool computer vision raise security concerns

#75

A paper came out that explains this effect and a method of minimizing it: http://arxiv.org/abs/1412.6572 Basically neural networks and many other machine learning methods are highly linear and continuous. So changing an input just slightly should change the output just slightly. If you change all of the inputs slightly in just the right directions, you can manipulate the output arbitrarily. These images are highly op…

> Basically neural networks and many other machine learning methods are highly linear No they're not! You introduce non-linearities like the sigmoid or tanh to make them highly non-linear.

Everything non-linear is linear to a first-order Taylor expansion. Hence why you can evolve small perturbations, the same thing is used for explicit numerical integration of non-linear equations.

Re: Images that fool computer vision raise security concerns

#76

Earlier quoted context omitted.

The problem is that you can't get very far into the discussion before all the available options involve overhauling the entire system. And there's no way, realistically, to do that.

> And there's no way, realistically, to do that. Yes, yes there is. We start with the most victimless crimes (drug possession for personal consumption) and go from there. Police unions will fight it. Prison unions will fight it. The court system will fight it. Nevertheless, it must be done.

None of what you just said is an argument against "And there's no way, realistically, to do that." which many people (myself included) believe when you're talking about a systematic overhaul of the entire criminal justice system. Saying "oh well it must be done" doesn't negate the fact that practically speaking, it's impossible.

Re: Images that fool computer vision raise security concerns

#77
post #27

Earlier quoted context omitted.

This always frustrates me when discussions of plea bargaining and the right to trial come up, and the argument is given that plea bargaining is a necessity because the courts would be horribly overloaded if every case went to trial. If the system doesn't have the resources to give every accused criminal a fair trial, then either you're making too many criminals, the system doesn't have enough resources, or both. Bypa…

> you're making too many criminals 80% of property crime goes unsolved, so unless you legalize vandalism and theft there are going to be criminals.

"Unsolved" is very different from "unprosecuted". The question here is why the courts are overloaded, not why there isn't additional load due to the 80% of property crimes you mention.

Re: Images that fool computer vision raise security concerns

#78
post #76

Earlier quoted context omitted.

> And there's no way, realistically, to do that. Yes, yes there is. We start with the most victimless crimes (drug possession for personal consumption) and go from there. Police unions will fight it. Prison unions will fight it. The court system will fight it. Nevertheless, it must be done.

None of what you just said is an argument against "And there's no way, realistically, to do that." which many people (myself included) believe when you're talking about a systematic overhaul of the entire criminal justice system. Saying "oh well it must be done" doesn't negate the fact that practically speaking, it's impossible.

We start with the most victimless crimes (drug possession for personal consumption) and go from there.

Did I not call that out enough? My apologies if so. Stop wasting time prosecuting what doesn't provide value in prosecuting.

Re: Images that fool computer vision raise security concerns

#79

This is some excellent research! It reminds me of the CV dazzle anti-facial-recognition makeup that made the rounds a while ago: http://www.theatlantic.com/features/archive/2014/07/makeup/3... This definitely reinforces my belief that having humans in the loop is not only desirable but necessary. For the majority of human history minus a few years you could only be accused of a crime by another human being. I'd like…

> If everyone is breaking so many laws that the police and courts can't keep up it doesn't mean that humanity is broken. It means that the law has gotten so far out of sync with humanity that the law is broken. People make the laws, not the other way around. The world would be a much better place if more people realized this.

The fact that can and does occur means the ideas of laws are broken.

But that's just the anarchist in me speaking.

Re: Images that fool computer vision raise security concerns

#80
post #74
post #27

Earlier quoted context omitted.

This always frustrates me when discussions of plea bargaining and the right to trial come up, and the argument is given that plea bargaining is a necessity because the courts would be horribly overloaded if every case went to trial. If the system doesn't have the resources to give every accused criminal a fair trial, then either you're making too many criminals, the system doesn't have enough resources, or both. Bypa…

Why should every crime go to trial? If the offender says "Fair cop, guv, you've got me bang to rights", why waste everyone's time proving that (s)he did it? It's basically the same as setting up your conditions to take advantage of short-circuit evaluation. You don't put the time-consuming and resource-hungry part first. Were there even the slimmest chance of acquittal, few defendants would utter that phrase without…

> If there's a suspicion of false confession, meaning that the suspect may not be the offender, that should normally be sorted out before court

How? Court is the process for sorting that out and assessing whether there is doubt, and the extent of that doubt.

Post reply on HN