Images that fool computer vision raise security concerns • Hacker News Board

Re: Images that fool computer vision raise security concerns

#171

Post by yosinski » Mon, Mar 23, 2015, 9:03 PM UTC

JoeAltmaier wrote: post #154

yosinski wrote: post #149

Earlier quoted context omitted.

No, but we did make use of (1) a large number of input -> network -> output iterations, along with (2) precisely measured output values to decide which input to try next. It may not be so easy to experiment in the same way on natural organisms (ethically or otherwise). Of course, if you're as clever as Tinbergen, you might be able to come up with patterns that fool organisms even without (1) or (2): https://imgur.com…

Perhaps a single experiment on millions of different people? A web experiment of some kind? "Which image looks more like a panda?" and flash two images on the screen.

That's a good idea, though note that there's a difference between asking "Which of these two images looks more like a panda?" and "Which of these two images looks more like a panda than a dog or cat?". The latter is the supervised learning setting used in the paper, and generally could lead to examples that look very different than pandas, as long as they look slightly more like pandas than dogs or cats. The former method is more like unsupervised density learning and could more plausibly produce increasingly panda-esque images over time.

A sort of related idea was explored with this site, where millions (ok, thousands) of users evolve shapes that look like whatever they want, but likely with a strong bias toward shapes recognizable to humans. Over time, many common motifs arise:

http://endlessforms.com/

Re: Images that fool computer vision raise security concerns

#172

Post by ars » Mon, Mar 23, 2015, 9:05 PM UTC

ChuckMcM wrote: post #131

One of the interesting things was the 'white noise' which was identified as various animals. I reminded me of people looking at noise and "seeing" data. Which for me suggests that at some level this isn't completely an artifact. If the algorithms developed are so closely modeled on human perception are susceptible to this sort of thing, humans probably are too. Perhaps that explains reports of people seeing things in…

The difference is that humans know they are seeing noise, they don't claim 99% confidence in their image, but rather a very low confidence.

Re: Images that fool computer vision raise security concerns

#173

Post by kingkawn » Mon, Mar 23, 2015, 9:12 PM UTC

jeffclune wrote: post #156

kingkawn wrote: post #61

Maybe this is the algorithm making art

See above. These images were accepted to a juried art competition and posted in a museum: http://www.evolvingai.org/share/20150213_184537.jpg

The awards only acknowledge, not express, what it has achieved.

The algorithm would be useful if it could generate/identify images that feel as emotionally transformative as good art.

Re: Images that fool computer vision raise security concerns

#174

Post by jessaustin » Mon, Mar 23, 2015, 9:21 PM UTC

toomuchtodo wrote: post #169

jessaustin wrote: post #166

Earlier quoted context omitted.

So you're advocating some sort of violent overthrow? We can't just snap our fingers and remake the political system into something completely different than it is, no matter how much Lawrence Lessig wishes we could. Grim fanatical purity is good for fundraising, but it won't get any laws passed.

No, I'm advocating for politicians to make decisions that are better for society, but will cause them to not be re-elected (similar to what happened in Australia when they outlawed firearms).

So you want the whole system to be based on the assumption that human beings are good? That is every bit as realistic as the assumption that USA is identical to Australia. b^)

Re: Images that fool computer vision raise security concerns

#175

Post by julianz » Mon, Mar 23, 2015, 9:24 PM UTC

toomuchtodo wrote: post #169

jessaustin wrote: post #166

Earlier quoted context omitted.

So you're advocating some sort of violent overthrow? We can't just snap our fingers and remake the political system into something completely different than it is, no matter how much Lawrence Lessig wishes we could. Grim fanatical purity is good for fundraising, but it won't get any laws passed.

No, I'm advocating for politicians to make decisions that are better for society, but will cause them to not be re-elected (similar to what happened in Australia when they outlawed firearms).

I agree with you, but the Port Arthur massacre happened 8 weeks into Howard's term as PM of Australia. The Howard government was in power for 10 years after that, they won four straight elections.

Re: Images that fool computer vision raise security concerns

#176

Post by yutah » Mon, Mar 23, 2015, 9:36 PM UTC

toomuchtodo wrote: post #11

msandford wrote: post #3

This is some excellent research! It reminds me of the CV dazzle anti-facial-recognition makeup that made the rounds a while ago: http://www.theatlantic.com/features/archive/2014/07/makeup/3... This definitely reinforces my belief that having humans in the loop is not only desirable but necessary. For the majority of human history minus a few years you could only be accused of a crime by another human being. I'd like…

> If everyone is breaking so many laws that the police and courts can't keep up it doesn't mean that humanity is broken. It means that the law has gotten so far out of sync with humanity that the law is broken. People make the laws, not the other way around. The world would be a much better place if more people realized this.

I think it is this way for 2 reasons... one, for-profit prisons and two, the ability for the government to put anyone away for a while

Re: Images that fool computer vision raise security concerns

#177

Post by toomuchtodo » Mon, Mar 23, 2015, 9:59 PM UTC

logfromblammo wrote: post #170

chc wrote: post #129

Earlier quoted context omitted.

Play it out in your head. An elected official declares he wants to try and starve a powerful and relatively popular union. What happens next?

Are we talking about Wisconsin governor Scott Walker? Because he survives a recall vote and then goes on to help make Wisconsin a right-to-work state. But wait. He specifically exempted police unions from his busting legislation, didn't he?

And after giving away massive tax breaks, is now pleading that the budget can't be balanced.

http://www.washingtonpost.com/blogs/govbeat/wp/2015/02/18/sc...

> To help close the state’s $283 million budget shortfall this year, Wisconsin Governor Scott Walker (R) plans to skip a $108 million debt payment scheduled for May.

> By missing the May payment, Walker will incur about $1.1 million in additional interest fees between 2015 and 2017. The $108 million debt will continue to live on the books; Walker’s budget proposal for 2015-2017 will pay down no more than about $18 million of the principal.

> In March last year, Walker signed a $541 million tax cut for both families and businesses. At that point, Wisconsin was facing a $1 billion budget surplus through June 2015, the Journal Sentinel reported.

Re: Images that fool computer vision raise security concerns

#178

Post by tormeh » Mon, Mar 23, 2015, 10:04 PM UTC

mikeash wrote: post #158

dragonwriter wrote: post #144

Earlier quoted context omitted.

> In the other direction, smoking weed in private harms nobody except the smoker. This is precisely the point that is contention. The entire argument for prohibiting marijuana is that this is, in fact, not the case, and that, through a number of indirect channels, people "smoking weed in private" harms others throughout society in a variety of ways. Obviously, as I said, there is considerable disagreement about wheth…

I still don't see any connection. Lots of what I would call "victimless crimes" are outlawed because of what the proponents of criminalization see as focused harm. Prostitution, for example, is seen as either harming the prostitute, or harming the patron's family. For drugs, it's often considered that the harm is focused on the user, not diffuse. And again, lots of crimes with diffuse harm are generally agreed on to…

The theory goes that the drugs impact the user and since the user becomes a brain damaged criminal afterwards, society will eventually need to deal with the user's shit.

There is some truth to this for some drugs, but not even close to most.

Re: Images that fool computer vision raise security concerns

#179

Post by aclinnovator » Mon, Mar 23, 2015, 10:08 PM UTC

I think that the major problem with CV is that it only recognizes images in isolation from each-other. Humans understand what they are looking at by finding the concept that lies at the intersection of all the small ideas in the image. For example, a human would recognize a keyboard because it contains a "means of input" on which there are "symbols" specifically the "alphabet", arranged in a "logical format"("QWERTYUIOP") which they know is the sign of a keyboard. If a human were to see a keybaord that looks different from most, they can still make the inference that it is a keyboard by understanding the underlying concepts of what they see.

On the other hand, a computer mechanically relates the specific format of a keyboard to the word "keyboard." It fuzzy matches the pixels of images to extract the object in the image: not the individual ideas implicit in the image.

Computer Vision needs more depth to actually be considered vision.

Re: Images that fool computer vision raise security concerns

#180

Post by tcfunk » Mon, Mar 23, 2015, 10:16 PM UTC

mikeash wrote: post #114

dragonwriter wrote: post #100

Earlier quoted context omitted.

So-called "victimless crimes" are crimes where the harm perceived is a diffuse harm rather than a focused harm on a particular individual. There's really no reason why diffuse harms should not be criminal while focused ones should (arguably, diffuse harms are less able to be addressed by individual action through the civil justice system, so outside of focused harms where the victim is unable to pursue action after t…

I've never seen "victimless crime" defined as diffuse versus focused harm, and I would strongly disagree with describing it as such. For example, dumping mercury into a major river causes extremely diffuse harm, but I'd never describe it as a "victimless crime." In the other direction, smoking weed in private harms nobody except the smoker. The harm is so focused it doesn't even touch anyone besides the offender. Yet…

I think the "diffuse harm" in the case of smoking weed in private is not talking about the act of smoking and who is or is not affected by the chemicals involved, etc.

It is rather looking at the overall picture of "What are the consequences of allowing weed to be bought/sold/grown. Is there harm in it?" The whole infrastructure, not just the end user.