Adversarial Learning for Good: On Deep Learning Blindspots
blog.kjamistan.com
Adversarial Learning for Good: On Deep Learning Blindspots
1–10 of 13 posts
Re: Adversarial Learning for Good: On Deep Learning Blindspots
#2Re: adversarial ML & steganography. There were 2 papers at this year's NIPS studying these ideas (http://papers.nips.cc/paper/6802-hiding-images-in-plain-sigh... and http://papers.nips.cc/paper/6791-generating-steganographic-i...)
Re: Adversarial Learning for Good: On Deep Learning Blindspots
#3I think it would be beneficial to explicitly mention model extraction attacks since they (kind of) enable such attacks: https://news.ycombinator.com/item?id=12557782
Re: Adversarial Learning for Good: On Deep Learning Blindspots
#4Video can be found here: https://media.ccc.de/v/34c3-8860-deep_learning_blindspots
Re: Adversarial Learning for Good: On Deep Learning Blindspots
#5The video example of a turtle being misclassified as a rifle is pretty scary https://www.youtube.com/watch?v=piYnd_wYlT8
We're approaching a world where AI will be more and more relied upon in dangerous situations. Imagine someone getting killed for something ridiculous like inadvertently holding an adversarial example. Public trust would have a hard time recovering.
Re: Adversarial Learning for Good: On Deep Learning Blindspots
#6I wonder how sensitive these results are. I mean, if you run one more learning round of the network, does the rifle turn to turtle in the eyes of the network immediately so that you would need to generate a new turtle for every single network? Or does that turtle look like rifle for all current neural networks? Or, most likely, somewhere in between?
Re: Adversarial Learning for Good: On Deep Learning Blindspots
#7The video example of a turtle being misclassified as a rifle is pretty scary https://www.youtube.com/watch?v=piYnd_wYlT8 We're approaching a world where AI will be more and more relied upon in dangerous situations. Imagine someone getting killed for something ridiculous like inadvertently holding an adversarial example. Public trust would have a hard time recovering.
>Imagine someone getting killed for something ridiculous like inadvertently holding an adversarial example.
This would never be the case with humans, right? No.
Re: Adversarial Learning for Good: On Deep Learning Blindspots
#8The video example of a turtle being misclassified as a rifle is pretty scary https://www.youtube.com/watch?v=piYnd_wYlT8 We're approaching a world where AI will be more and more relied upon in dangerous situations. Imagine someone getting killed for something ridiculous like inadvertently holding an adversarial example. Public trust would have a hard time recovering.
Would adding random noise to an image before running it through the classifier mitigate these kinds of attacks?
Re: Adversarial Learning for Good: On Deep Learning Blindspots
#9I wonder how sensitive these results are. I mean, if you run one more learning round of the network, does the rifle turn to turtle in the eyes of the network immediately so that you would need to generate a new turtle for every single network? Or does that turtle look like rifle for all current neural networks? Or, most likely, somewhere in between?
If you don't change the data, why would the network change its results with just some more learning iterations? It's already converging.
Re: Adversarial Learning for Good: On Deep Learning Blindspots
#10The video example of a turtle being misclassified as a rifle is pretty scary https://www.youtube.com/watch?v=piYnd_wYlT8 We're approaching a world where AI will be more and more relied upon in dangerous situations. Imagine someone getting killed for something ridiculous like inadvertently holding an adversarial example. Public trust would have a hard time recovering.
Would adding random noise to an image before running it through the classifier mitigate these kinds of attacks?
No, ironically because classifiers are trained to be robust to noise. Adversarial examples are generated by altering the original in one specific dimension of a high-dimensional vector space. Random noise is vanishingly unlikely to undo that transformation, so the classification will stay wrong.