Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How Adversarial Attacks Work

blog.ycombinator.com

Post reply on HN
Print view View original

21–30 of 148 posts

Re: How Adversarial Attacks Work

#21

Post by neptvn »

> Apart from the fact that nobody wants to risk having false positives, there’s a simple argument as old as machine learning itself: whatever a human can do, a machine can be taught to do. Humans have no problem correctly interpreting adversarial examples, so there must be a way to do it automatically.

So by all means, the authors (and a vast majority of researchers) seem to be confident that ML/DL is the road to AGI, hence can "solve" human intelligence (given it is computational)? For how long are we gonna drag the adage that mimicking a human (Turing test) is equal to reaching human levels of intelligence?

Re: How Adversarial Attacks Work

#22

Post by danharaj »

munificent wrote: post #17
yters wrote: post #4

Ah, this is why AlphaGo won't release their source code or models. If all ML algorithms can be fooled so trivially, this shows the human mind is not an ML algorithm.

I think your logic is: 1. All ML algorithms can be fooled trivially. 2. The human mind cannot be fooled trivially. 3. Therefore, the human mind is not an ML algorithm. But claim number 2 is clearly wrong. Human minds are trivially fooled. Here's one: http://www.jimonlight.com/wp-content/uploads/2012/02/Paralle... This is exactly what an optical illusion is.

Cool illusion; I figured it out and so will most humans. I don't think the claim that these sorts of illusions are of the same nature as, say, single pixel attacks against neural networks is justified. On the other hand, it is also true that it's not clear whether such attacks on our current crop of primitive neural networks will work on more advanced autonomous systems that regulate themselves.

Re: How Adversarial Attacks Work

#23

Post by joe_the_user »

alexbeloi wrote: post #19
joe_the_user wrote: post #5

"Recent studies by Google Brain have shown that any machine learning classifier can be tricked to give incorrect predictions" -- That has to be an overly broad statement (edit: it might true if you say "neural nets" or something specific instead). I would assume they mean any standard deep learning system and maybe any system that is more or less "generalized regression" but that couldn't be "any machine learning sys…

>I mean one could imagine a deterministic model that couldn't be "tricked". The models in reference are deterministic, most models for classification/regression are deterministic. Adversarial attacks are showing that the models are chaotic (in the dynamical systems sense), they're very sensitive to their inputs. edit: Not all models have this issue, it's been shown that all the typical image based convolutional neura…

Yeah, my complaint is that they're using the generic phrase "machine learning" for specific deep learning and other neural net methods. Certainly, a wide class of "black box" algorithms can be fooled, mostly because they extrapolate in surprising and paradoxical ways.

Re: How Adversarial Attacks Work

#24

Post by moxious »

neptvn wrote: post #21

> Apart from the fact that nobody wants to risk having false positives, there’s a simple argument as old as machine learning itself: whatever a human can do, a machine can be taught to do. Humans have no problem correctly interpreting adversarial examples, so there must be a way to do it automatically. So by all means, the authors (and a vast majority of researchers) seem to be confident that ML/DL is the road to AGI…

It is not true that whatever a human can do, a machine can be taught to do. The human must have insight into HOW they do it in order to teach it, or otherwise come up with some new algorithm. There are a large class of things humans do that they don't understand the mechanics behind, and for which there also aren't algorithms.

I'm not talking empathy or philosophy. How about just folding laundry. Not just one type, not in a controlled environment, but folding any laundry anywhere.

Re: How Adversarial Attacks Work

#26

Post by mLuby »

Am I the only one who thinks "adversarial attack" is both a redundant and unhelpful name?

- Redundant: Anyone who attacks you is by definition your adversary.

- Unhelpful: According to the article, "designing an input in a specific way to get the wrong result from the model is called an adversarial attack." That sounds much closer to spoofing attack ("a situation in which one person or program successfully masquerades as another by falsifying data" -Wikipedia). For example, a turtle masquerades as a gun by spoofing the machine learning system by changing irrelevant visual details.

Re: How Adversarial Attacks Work

#27

Post by tlb »

mLuby wrote: post #26

Am I the only one who thinks "adversarial attack" is both a redundant and unhelpful name? - Redundant: Anyone who attacks you is by definition your adversary. - Unhelpful: According to the article, "designing an input in a specific way to get the wrong result from the model is called an adversarial attack." That sounds much closer to spoofing attack ("a situation in which one person or program successfully masquerade…

Adversarial means something specific in the ML community. https://en.wikipedia.org/wiki/Adversarial_machine_learning

Re: How Adversarial Attacks Work

#28

Post by scrooched_moose »

Is there any reason to think this would work at all in the real world? All of these "attacks" require complete control of the image being fed to the classifier.

In the ATM example you don't directly load an image of the check to the computer inside the machine. You design a check in photoshop, add the noise, print it out, feed it to the machine, which takes a picture of the check. Mobile bank apps still require you to take a picture of the check so you don't have enough control there either.

Similarly in the road sign example; the lighting, angle between car and sign, dirt on the sign, etc all mean the car sees a much different image than you designed.

I'd think all of these steps mean the classifier gets a dramatically different image than you intend and the attack fails. There's maybe a vanishingly small probability it works when the stars align, but that could be easily mitigated by taking multiple consecutive images and looking for an odd results.

Re: How Adversarial Attacks Work

#29

Post by yters »

munificent wrote: post #17
yters wrote: post #4

Ah, this is why AlphaGo won't release their source code or models. If all ML algorithms can be fooled so trivially, this shows the human mind is not an ML algorithm.

I think your logic is: 1. All ML algorithms can be fooled trivially. 2. The human mind cannot be fooled trivially. 3. Therefore, the human mind is not an ML algorithm. But claim number 2 is clearly wrong. Human minds are trivially fooled. Here's one: http://www.jimonlight.com/wp-content/uploads/2012/02/Paralle... This is exactly what an optical illusion is.

In order for optical illusions to be a counter example, I must believe those lines are not parallel with near certainty. While I perceive them as such, I don't believe they are. The fact that I can know my perception does not match reality is what gives optical illusions their novelty. Thus, the very nature of optical illusions makes them not a counter example.

A true counter example must be similar to making me believe a car is a toaster by manipulating a single cone cell in my eye. Such a possibility is absurd, which leads me to conclude the human mind is not ML.

Re: How Adversarial Attacks Work

#30

Post by anishathalye »

scrooched_moose wrote: post #28

Is there any reason to think this would work at all in the real world? All of these "attacks" require complete control of the image being fed to the classifier. In the ATM example you don't directly load an image of the check to the computer inside the machine. You design a check in photoshop, add the noise, print it out, feed it to the machine, which takes a picture of the check. Mobile bank apps still require you t…

Goodfellow et al. demonstrate that adversarial misclassification works in the physical world: https://arxiv.org/abs/1607.02533 -- a printed image is consistently misclassified (demo here: https://www.youtube.com/watch?v=zQ_uMenoBCk)

Recent work by me and some friends demonstrates that physical-world adversarial examples can actually be made quite robust, and you can synthesize 3D adversarial objects as well, and make them consistently classify as a desired target class: http://www.labsix.org/physical-objects-that-fool-neural-nets...

Post reply on HN