Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Adversarial Attacks Work

blog.ycombinator.com

Post reply on HN
Print view View original

111–120 of 148 posts

Re: How Adversarial Attacks Work

#111

Post by blauditore »

It seems like all adversarial examples I've seen so far are enabled by networks overfitting to small-scale features. By overfitting I mean that they give more importance to details compared to humans, who recognize much more the image as a whole.

Maybe the problem could be mitigated by penalizing "direct" responses to small-scale features while training, which is not a trivial thing to do though. One approach I could think of is training with multiple altered versions of the image, e.g. various amounts of blur, noise and mean/median filters applied. Or the other way around: To be more confident about a result, scale down the image to a fraction of it's size, run detection on that and compare results.

Are such techniques in use, or being researched on? I'm not too much in the loop about those topics.

Re: How Adversarial Attacks Work

#113

Post by IanCal »

moxious wrote: post #24
neptvn wrote: post #21

> Apart from the fact that nobody wants to risk having false positives, there’s a simple argument as old as machine learning itself: whatever a human can do, a machine can be taught to do. Humans have no problem correctly interpreting adversarial examples, so there must be a way to do it automatically. So by all means, the authors (and a vast majority of researchers) seem to be confident that ML/DL is the road to AGI…

It is not true that whatever a human can do, a machine can be taught to do. The human must have insight into HOW they do it in order to teach it, or otherwise come up with some new algorithm. There are a large class of things humans do that they don't understand the mechanics behind, and for which there also aren't algorithms. I'm not talking empathy or philosophy. How about just folding laundry. Not just one type, n…

> It is not true that whatever a human can do, a machine can be taught to do. The human must have insight into HOW they do it in order to teach it, or otherwise come up with some new algorithm.

Why does this mean a machine cannot be taught to do it?

> There are a large class of things humans do that they don't understand the mechanics behind,

Sure.

> and for which there also aren't algorithms.

Well, we manage to encode it in our brains.

Re: How Adversarial Attacks Work

#114

Post by stonewhite »

ChuckMcM wrote: post #81
Houshalter wrote: post #80

Earlier quoted context omitted.

I mean people can already do that to human drivers. You can easily remove a stop sign. Or put a sticker over an important sign. Any human driver will be fooled! Yet this doesn't seem to be a prevalent problem.

And they have: http://www.nytimes.com/1997/06/21/us/3-are-sentenced-to-15-y... It just makes it harder to catch who did it.

Not relevant to the discussion, but turns out these kids didn't steal those signs.

https://www.law.umich.edu/special/exoneration/Pages/casedeta...

Re: How Adversarial Attacks Work

#116

Post by jpttsn »

YeGoblynQueenne wrote: post #73
jpttsn wrote: post #65

Earlier quoted context omitted.

https://m.youtube.com/watch?v=A4QcyW-qTUg

Again, this is an optical illusion that fools depth perception. It doesn't make you see the T-Rex as a tea-pot, say. Different things.

Fair point; it's not exactly the same mistake. If humans made those, this discussion would probably never happen.

What I tried to rebut is your first, weaker assertion:

> The problem with optical illusions like that is that they are, in their vast majority, made of abstract shapes. Most of them play with our perception of distance and depth - and the majority again work on two dimensions, only.

This makes me see a moving 3d object when I am in fact looking at a static flat object; and I can't shake it, even if I try.

Re: How Adversarial Attacks Work

#117

Post by SomeStupidPoint »

smogcutter wrote: post #54
projectileboy wrote: post #43

Earlier quoted context omitted.

One possible mitigation could be the use of geocoded roadway metadata. I rented a car in France that had this - it would let me know when I was exceeding the speed limit based on my position. Not hyper-accurate, and so probably not a replacement for all road signs, but certainly good enough for many situations.

Was thinking of this too, but what happens when the map and "ground truth" disagree? The machine would need to be able to identify the rare circumstances when it should trust the map over its senses. If it's capable of that, this wouldn't be a problem in the first place.

When driving, that probably doesn't really matter:

If the map and road sign disagree, just do what everyone around you is doing (ie, go with the flow) until you recover agreement. It's not perfect, but stretches out attacks to having to compromise long stretches of road and multiple vehicle types. If the cars for the next mile somewhere can find pace, the whole group can; similarly, if any kind of car can find pace, they can all use that to set.

Obviously safety checks and such, but human drivers only loosely follow signs and often get behavioral cues from other cars -- why can't automated cars do the same when they get confused?

Re: How Adversarial Attacks Work

#118

Post by naveen99 »

haraldurt wrote: post #86
naveen99 wrote: post #71

Earlier quoted context omitted.

Thanks. But they still need to be able to use the black box a brute force number of times on shady inputs and get detailed outputs during their gradient descent. Not going to be allowed on a check reader or anything sensitive.

Not necessarily. Adversarial examples have been shown to, for instance, be transferable across different networks with different hyperparameters (e.g., number of layers) trained on disjoint subsets of a training set [0, section 4.2]. There are more references from the paper linked by the OP. [0] https://arxiv.org/abs/1312.6199

Thanks. I wonder if adversarial training helps prevent overfitting too. Could you use adversarial training to beat alphago ?

Re: How Adversarial Attacks Work

#120

Post by rahimnathwani »

blauditore wrote: post #111

It seems like all adversarial examples I've seen so far are enabled by networks overfitting to small-scale features. By overfitting I mean that they give more importance to details compared to humans, who recognize much more the image as a whole. Maybe the problem could be mitigated by penalizing "direct" responses to small-scale features while training, which is not a trivial thing to do though. One approach I could…

"training with multiple altered versions of the image, e.g. various amounts of blur, noise and mean/median filters applied"

"Are such techniques in use"

Yes, it's called 'data augmentation': https://medium.com/towards-data-science/deep-learning-3-more...

Post reply on HN