Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Images that fool computer vision raise security concerns

news.cornell.edu

Post reply on HN
Print view View original

111–120 of 220 posts

Re: Images that fool computer vision raise security concerns

#111

Post by deeviant »

I know many here know this already but I'll say it anyway since I ran into people with a misconception related to this: the particular misidentifications are specific to the algorithm and training set used, it's not like all computers recognize those blocks as cheetah or what not.

We use ML-based computer vision at my work, so I have a bit of experience here. I think the biggest practice take away from observation the ML can give some wonky results is that ML system can be a real PITA to debug.

Re: Images that fool computer vision raise security concerns

#112

Post by clort »

platz wrote: post #94
drpgq wrote: post #70

Earlier quoted context omitted.

That's really not that interesting. I work as a research scientist at a face recognition company. If you want, you can detect faces at any angle, but sometimes for speed you will only check upright faces.

it might not be interesting to you as a researcher, but to me as a participant in in a world increasingly filled with face recognition products it's worth knowing that different products will have different capabilities based on their "speed" requirements.

I am not a researcher, but I would suggest that perhaps you would find that in the case where a crowd was being parsed in realtime (looking out for somebody so we can catch them before they leave the station for example), the "speed" dial would be turned up as high as necessary, but that the video stream would be stored and the data set reprocessed at a later time with it at a better setting (so you can still say "yes she was here at x time, though we missed her")

in other words, the bandwidth of the image recognition product is not the same as the bandwidth of the video storage.

Re: Images that fool computer vision raise security concerns

#113

Post by jpeterson »

c-slice wrote: post #105

This isn't very groundbreaking, they're simply discovering the inherent weaknesses in neural net learning. We've know for years that neural nets have inherent gaps in their training and this is just taking advantage of that. This isn't really an issue in a real life scenario as you wouldn't be able to determine without thousands of recursions which images produce faulty results.

Well, obviously they should've checked with you before publishing their paper.

Re: Images that fool computer vision raise security concerns

#114

Post by mikeash »

dragonwriter wrote: post #100
mellavora wrote: post #93

Earlier quoted context omitted.

In support of your position, how did we ever let people convince us that victimless "crimes" were crimes? The definition of crime rests on the concept of harm, which requires a victim.

So-called "victimless crimes" are crimes where the harm perceived is a diffuse harm rather than a focused harm on a particular individual. There's really no reason why diffuse harms should not be criminal while focused ones should (arguably, diffuse harms are less able to be addressed by individual action through the civil justice system, so outside of focused harms where the victim is unable to pursue action after t…

I've never seen "victimless crime" defined as diffuse versus focused harm, and I would strongly disagree with describing it as such.

For example, dumping mercury into a major river causes extremely diffuse harm, but I'd never describe it as a "victimless crime."

In the other direction, smoking weed in private harms nobody except the smoker. The harm is so focused it doesn't even touch anyone besides the offender. Yet this is almost the canonical example of "victimless crime."

It's not about diffuse harm, it's about whether anyone was harmed at all besides the accused.

Re: Images that fool computer vision raise security concerns

#115

Post by trhway »

wmil wrote: post #110
mikeash wrote: post #27

Earlier quoted context omitted.

This always frustrates me when discussions of plea bargaining and the right to trial come up, and the argument is given that plea bargaining is a necessity because the courts would be horribly overloaded if every case went to trial. If the system doesn't have the resources to give every accused criminal a fair trial, then either you're making too many criminals, the system doesn't have enough resources, or both. Bypa…

This is a big issue in the US and it actually goes back to the Warren court. They issued a long series of rulings making it difficult to prosecute cases, without worrying about the consequences. By the 70s crime had skyrocketed and it was clear that they had gone too far. But instead of issuing a mea culpa and reexamining past rulings, the various courts started allowing prosecutors to claim broad new powers and take…

>By this point there's no real way to fix it. The legal system is based strongly on precedent. It can't just undo major rulings of the past and replace the with something sane.

sounds exactly like the point where a major fix must be done. Life changes faster and faster. The precedent system was great in 15th century when things look pretty much the same like in the 14th or 13th century.

Re: Images that fool computer vision raise security concerns

#116

Post by bediger4000 »

FLGMwt wrote: post #13

Who decided to name it a "lesser panda"? And of the alternative names for a red panda, firefox, or red cat-bear why would someone choose "lesser panda"?

That is a question for the ages. Who decided on "cool ranch flavor" for Doritos? And what's a "cool ranch"? Or "blue raspberry", a common flavor for candy these day, or "TV Spokesmodel" (https://www.google.com/?gws_rd=ssl#q=TV+spokesmodel). There's lots of weird-beard names that get applied to stuff, and nobody has a say in those names.

Re: Images that fool computer vision raise security concerns

#117

Post by joe_the_user »

wmil wrote: post #110
mikeash wrote: post #27

Earlier quoted context omitted.

This always frustrates me when discussions of plea bargaining and the right to trial come up, and the argument is given that plea bargaining is a necessity because the courts would be horribly overloaded if every case went to trial. If the system doesn't have the resources to give every accused criminal a fair trial, then either you're making too many criminals, the system doesn't have enough resources, or both. Bypa…

This is a big issue in the US and it actually goes back to the Warren court. They issued a long series of rulings making it difficult to prosecute cases, without worrying about the consequences. By the 70s crime had skyrocketed and it was clear that they had gone too far. But instead of issuing a mea culpa and reexamining past rulings, the various courts started allowing prosecutors to claim broad new powers and take…

By the 70s crime had skyrocketed and it was clear that they had gone too far.

Any claim that court ruler enabled crime is highly debatable. Factors causing increases and decreases in crime rates are difficult to pin-point.

One argument is returning Vietnam veterans lead to the increase in violent crime - post-war eras have a history of being associated with crime-increases from the return of veterans habituated to violence.

Re: Images that fool computer vision raise security concerns

#118

Post by joe_the_user »

toomuchtodo wrote: post #11
msandford wrote: post #3

This is some excellent research! It reminds me of the CV dazzle anti-facial-recognition makeup that made the rounds a while ago: http://www.theatlantic.com/features/archive/2014/07/makeup/3... This definitely reinforces my belief that having humans in the loop is not only desirable but necessary. For the majority of human history minus a few years you could only be accused of a crime by another human being. I'd like…

> If everyone is breaking so many laws that the police and courts can't keep up it doesn't mean that humanity is broken. It means that the law has gotten so far out of sync with humanity that the law is broken. People make the laws, not the other way around. The world would be a much better place if more people realized this.

Well, I think a large percentage of people do realize this.

They just forget it as soon as the news media present whatever is the current horror story.

Re: Images that fool computer vision raise security concerns

#119

Post by raynassar »

lawlessone wrote: post #68

Machine pareidolia? On a serious not couldn't we keep training the same DNN's using these white noise images as negative examples?

That point was covered in the article.

>In a further step, the researchers tried ?retraining? the DNN by showing it fooling images and labeling them as such. This produced some improvement, but the researchers said that even these new, retrained networks often could be fooled.

Re: Images that fool computer vision raise security concerns

#120

Post by TazeTSchnitzel »

Houshalter wrote: post #81
uncoder0 wrote: post #24

Not directly related but, I was at a security related convention and overheard some people talking about an image that when occupying <3/4 of a frame will crash any digital camera. (Phone, DSLR, IP Camera) Does anyone know anymore information about this image and effect? I imagine it's a bug in some low level firmware of a common IC for digital photography DSP but, I'm very unfamiliar with digital cameras. It also co…

Could be related to this? http://en.wikipedia.org/wiki/EURion_constellation

It's most likely not the constellation itself, but the other mechanism:

https://en.wikipedia.org/wiki/EURion_constellation#Other_ban...

In particular, have a look at this:

http://www.cl.cam.ac.uk/~sjm217/projects/currency/

Post reply on HN