Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Adversarial Attacks Work

blog.ycombinator.com

Post reply on HN
Print view View original

71–80 of 148 posts

Re: How Adversarial Attacks Work

#71

Post by naveen99 »

Winterflow3r wrote: post #49
naveen99 wrote: post #48

Don't you need access to the classifier internals to train the adversarial network ? Nobody is going to publish the network weights for a check reading machine...

No, this paper by Papernot et al shows how to do blackbox attacks without knowledge of model internals. https://arxiv.org/abs/1602.02697

Thanks. But they still need to be able to use the black box a brute force number of times on shady inputs and get detailed outputs during their gradient descent. Not going to be allowed on a check reader or anything sensitive.

Re: How Adversarial Attacks Work

#73

Post by YeGoblynQueenne »

jpttsn wrote: post #65
YeGoblynQueenne wrote: post #44

Earlier quoted context omitted.

The problem with optical illusions like that is that they are, in their vast majority, made of abstract shapes. Most of them play with our perception of distance and depth - and the majority again work on two dimensions, only. It's really hard to imagine an optical illusion that makes you mistake objects in the physical world for something else- say, panda for a lawn mower or a car for a pigeon, or something like tha…

https://m.youtube.com/watch?v=A4QcyW-qTUg

Again, this is an optical illusion that fools depth perception. It doesn't make you see the T-Rex as a tea-pot, say.

Different things.

Re: How Adversarial Attacks Work

#74

Post by YeGoblynQueenne »

munificent wrote: post #58
YeGoblynQueenne wrote: post #44

Earlier quoted context omitted.

The problem with optical illusions like that is that they are, in their vast majority, made of abstract shapes. Most of them play with our perception of distance and depth - and the majority again work on two dimensions, only. It's really hard to imagine an optical illusion that makes you mistake objects in the physical world for something else- say, panda for a lawn mower or a car for a pigeon, or something like tha…

> It's really hard to imagine an optical illusion that makes you mistake objects in the physical world for something else- say, panda for a lawn mower or a car for a pigeon, or something like that. Here's a physical object that makes you mistake an insect for a plant: https://en.wikipedia.org/wiki/Phasmatodea

What romaniv said- it's a different thing.

Re: How Adversarial Attacks Work

#75

Post by YeGoblynQueenne »

throwaway613834 wrote: post #55
YeGoblynQueenne wrote: post #44

Earlier quoted context omitted.

The problem with optical illusions like that is that they are, in their vast majority, made of abstract shapes. Most of them play with our perception of distance and depth - and the majority again work on two dimensions, only. It's really hard to imagine an optical illusion that makes you mistake objects in the physical world for something else- say, panda for a lawn mower or a car for a pigeon, or something like tha…

> It's really hard to imagine an optical illusion that makes you mistake objects in the physical world for something else- say, panda for a lawn mower or a car for a pigeon, or something like that. Sure but people do, for example, mistake each others' faces or voices. You don't need to mistake your friend for a lawnmower for it to be dangerous. Also, for example, I often mishear my own name when someone else is speak…

Well, people misidentify others when viewing conditions are poor or when they dont' know the other person well. For instance, say my mother was standing in front of me in broad daylight and I was given half a second to look at her face; I really don't see how I'd fail to identify here, unless her face had changed drastically for some reason.

So, to be fair, this too is a different thing than what we're talking about.

Re: How Adversarial Attacks Work

#76

Post by Houshalter »

aidenn0 wrote: post #72

ELI5: Why are adversarial attacks not preventable by adding unpredictable noise to untrusted inputs?

The attacks are resistant to noise, or at least can made to be so. If every single input is tweaked in exactly the right direction, noise won't undo that. Most inputs will still be pointing in the adversarial direction. The noise will move some inputs back to their original position, but others will be pushed even further into adversarial territory.

Re: How Adversarial Attacks Work

#78

Post by WalterSear »

aidenn0 wrote: post #72

ELI5: Why are adversarial attacks not preventable by adding unpredictable noise to untrusted inputs?

Because they aren't noise: they just seem like noise to humans.

If you add in noise, then you have to train the network to disregard that noise. And the adversarial input will then be features that occur above this noise floor you ignoring.

Re: How Adversarial Attacks Work

#79

Post by xstartup »

Well, this is getting more attention because it's important for the "ad network moderation". If Google/FB fail at moderation using these methods, they'll have to HIRE lots of humans to do it for them, which often involves contracting it to outsources like Accenture. This will put downward pressure on their billion dollar revenue. Humans are expensive but vastly effective at ad moderation.

Re: How Adversarial Attacks Work

#80

Post by Houshalter »

ChuckMcM wrote: post #34

This weakness is one that I think will plague self driving cars as sign recognition will be key and without some ability to insure that they cannot be dangerously fooled, it will be hard to get them certified. The canonical example is to make a no left turn sign recognize as a no right turn sign and have the car go the wrong way on a one way street. Clearly there is a marketing opportunity for t-shirts that make you…

I mean people can already do that to human drivers. You can easily remove a stop sign. Or put a sticker over an important sign. Any human driver will be fooled! Yet this doesn't seem to be a prevalent problem.
Post reply on HN