How Adversarial Attacks Work
blog.ycombinator.com
How Adversarial Attacks Work
1–10 of 148 posts
Re: How Adversarial Attacks Work
#2[deleted]
Re: How Adversarial Attacks Work
#3Very recently, it was found that changing a single, carefully placed, pixel is enough to confuse NN classifiers. https://arxiv.org/abs/1710.08864
Re: How Adversarial Attacks Work
#4Ah, this is why AlphaGo won't release their source code or models.
If all ML algorithms can be fooled so trivially, this shows the human mind is not an ML algorithm.
Re: How Adversarial Attacks Work
#5"Recent studies by Google Brain have shown that any machine learning classifier can be tricked to give incorrect predictions"
-- That has to be an overly broad statement (edit: it might true if you say "neural nets" or something specific instead). I would assume they mean any standard deep learning system and maybe any system that is more or less "generalized regression" but that couldn't be "any machine learning system", I mean one could imagine a deterministic model that couldn't be "tricked".
Plus a link would be good (it's not around the quote, I don't know if it's elsewhere in the article).
Re: How Adversarial Attacks Work
#6Does the adversarial attack require access to the model making the predictions?
It seems like the attack relies on `doping` the input with features present in a different target, or by masking features of the existing target.
Could you so specifically attack a target without knowledge of its features?
Re: How Adversarial Attacks Work
#7"Recent studies by Google Brain have shown that any machine learning classifier can be tricked to give incorrect predictions" -- That has to be an overly broad statement (edit: it might true if you say "neural nets" or something specific instead). I would assume they mean any standard deep learning system and maybe any system that is more or less "generalized regression" but that couldn't be "any machine learning sys…
It's probably referencing Szegedy et al. 2013 (https://arxiv.org/abs/1312.6199).
Re: How Adversarial Attacks Work
#8"Recent studies by Google Brain have shown that any machine learning classifier can be tricked to give incorrect predictions" -- That has to be an overly broad statement (edit: it might true if you say "neural nets" or something specific instead). I would assume they mean any standard deep learning system and maybe any system that is more or less "generalized regression" but that couldn't be "any machine learning sys…
No system is 100% secure. What they mean is you can train an adversarial input that would work with a deep learning system and this can fool other machine learning systems. Or that you can create a deep learning system that would fool another (you use a GAN to make forgeries to fool your target ).
Re: How Adversarial Attacks Work
#9I don’t buy any of the attacks listed here or see how the examples being imperceptible is actually a factor.
If you have the ability to modify the check why not make it actually look like it’s for a 1000000 dollars (ie even to a human).
If you are going to go out and replace speed limit signs to fool self driving cars, it’s probably equally dangerous whether or not the change is obvious, because if it’s way out of bounds a car won’t do it but if it isn’t then it would also fool humans.
Anyone have some more realistic attacks that are specifically possible due to the imperceptible nature of the changes in the input? Most of the harm I’ve heard in examples simply comes from the fact that if you are controlling the input to an ml system, out can get it to do whatever you want even without using any of these techniques: just actually change the class of the input.
Re: How Adversarial Attacks Work
#10"Recent studies by Google Brain have shown that any machine learning classifier can be tricked to give incorrect predictions" -- That has to be an overly broad statement (edit: it might true if you say "neural nets" or something specific instead). I would assume they mean any standard deep learning system and maybe any system that is more or less "generalized regression" but that couldn't be "any machine learning sys…
No system is 100% secure. What they mean is you can train an adversarial input that would work with a deep learning system and this can fool other machine learning systems. Or that you can create a deep learning system that would fool another (you use a GAN to make forgeries to fool your target ).
> No [machine learning] system is 100% secure.
That is what's being claimed without proof, when they meant "existing neural network systems are known to be insecure"