Was there a follow up to this paper?
NO Need to Worry about Adversarial Examples in Object Detection in Autonomous Vehicles
https://arxiv.org/abs/1707.03501
How Adversarial Attacks Work
91–100 of 148 posts
Re: How Adversarial Attacks Work
#92This weakness is one that I think will plague self driving cars as sign recognition will be key and without some ability to insure that they cannot be dangerously fooled, it will be hard to get them certified. The canonical example is to make a no left turn sign recognize as a no right turn sign and have the car go the wrong way on a one way street. Clearly there is a marketing opportunity for t-shirts that make you…
Or, simply with better machine learning.
> The border between “truth” and “false” is almost linear. The first cool thing we can derive from it is that when you follow the gradient, once you find the area where the predicted class changes, you can be fairly confident that the attack is successful. On the other hand, it tells us that the structure the decision function is far simpler that most researchers thought it to be.
Humans can be fooled by optical illusions as well; but those illusions are much more limited and much more noticeable than most of these. My (very non-specialist) interpretation of the italicize clause is that a) vulnerability to these attacks is a continuum, not a binary, and b) the current ease of these attacks reflects the crudity of our current ML techniques.
Re: How Adversarial Attacks Work
#93Re: How Adversarial Attacks Work
#94> Apart from the fact that nobody wants to risk having false positives, there’s a simple argument as old as machine learning itself: whatever a human can do, a machine can be taught to do. Humans have no problem correctly interpreting adversarial examples, so there must be a way to do it automatically. So by all means, the authors (and a vast majority of researchers) seem to be confident that ML/DL is the road to AGI…
It is not true that whatever a human can do, a machine can be taught to do. The human must have insight into HOW they do it in order to teach it, or otherwise come up with some new algorithm. There are a large class of things humans do that they don't understand the mechanics behind, and for which there also aren't algorithms. I'm not talking empathy or philosophy. How about just folding laundry. Not just one type, n…
It is not true that whatever a human can do, is the correct way to look at it. The fine motor skills and insight needed for many tasks will always be beyond that of a machine. A machine also has a great deal of trouble with adapting to things popping up that it has to deal with.
Re: How Adversarial Attacks Work
#95This weakness is one that I think will plague self driving cars as sign recognition will be key and without some ability to insure that they cannot be dangerously fooled, it will be hard to get them certified. The canonical example is to make a no left turn sign recognize as a no right turn sign and have the car go the wrong way on a one way street. Clearly there is a marketing opportunity for t-shirts that make you…
Alternately, if you have multiple models that should normally give the same result, this can apparently be used to automatically find test cases where they differ:
DeepXplore: automated whitebox testing of deep learning systems https://blog.acolyer.org/2017/11/01/deepxplore-automated-whi...
Re: How Adversarial Attacks Work
#96Earlier quoted context omitted.
> Although our models are more vulnerable to white-box FGSM samples compared to the v3adv model, ensemble adversarial training significantly increases robustness to black-box attacks that transfer FGSM samples crafted on the holdout Inception v4. So, I just train my new adversary on the "new" model that was trained on the previous adversarial examples. And now we're back to square one. I suspect the problem of advers…
I'm not sure how the quote supports your argument. Adversial examples generalize well accross many different classifiers. Shallow NN's can be fooled just as well, it seems to be more of a problem of linear models in general. Apparently Geoff Hintons Capsule Networks are more robust due to being "less linear" (Ian Goodfellow mentioned this in a recent talk, don't have the references now to back it up)
I'm not sure it's about how shallow the network is; even logistic regression can be fooled by the same techniques (e.g. 1-layer NN). That being said, maybe it does have something to do with linearity (I suspect not) or maybe it's just generally harder to deal with nonlinear functions.
Re: How Adversarial Attacks Work
#97The _only_ solution is coming up with a globally-accepted open source base set of instruction for any and all AI or AI-capable code.
Re: How Adversarial Attacks Work
#98Can such methods be used to "fingerprint" proprietary datasets by tainting them ? For example, i want to make sure that my dataset is not stolen and used by someone else (Waymo?). So I taint it using an adversarial method and create a "canary test set" that will uniquely identify if my dataset has been used in some training.
I believe google maps has fake streets as a canary for their maps being scraped...
Re: How Adversarial Attacks Work
#99Earlier quoted context omitted.
I think your logic is: 1. All ML algorithms can be fooled trivially. 2. The human mind cannot be fooled trivially. 3. Therefore, the human mind is not an ML algorithm. But claim number 2 is clearly wrong. Human minds are trivially fooled. Here's one: http://www.jimonlight.com/wp-content/uploads/2012/02/Paralle... This is exactly what an optical illusion is.
In order for optical illusions to be a counter example, I must believe those lines are not parallel with near certainty. While I perceive them as such, I don't believe they are. The fact that I can know my perception does not match reality is what gives optical illusions their novelty. Thus, the very nature of optical illusions makes them not a counter example. A true counter example must be similar to making me beli…
> the human mind is not ML
Pooling layers in CNNs throw away lots of information about spatial co-occurrence of features, which leads to a possibility of adversarial images where adversarial features are scattered all over the place and so they don't significantly affect humans' visual processing.
The conclusion should be "Human mind is not that kind of ML".
Re: How Adversarial Attacks Work
#100Earlier quoted context omitted.
In order for optical illusions to be a counter example, I must believe those lines are not parallel with near certainty. While I perceive them as such, I don't believe they are. The fact that I can know my perception does not match reality is what gives optical illusions their novelty. Thus, the very nature of optical illusions makes them not a counter example. A true counter example must be similar to making me beli…
> the human mind is not ML Pooling layers in CNNs throw away lots of information about spatial co-occurrence of features, which leads to a possibility of adversarial images where adversarial features are scattered all over the place and so they don't significantly affect humans' visual processing. The conclusion should be "Human mind is not that kind of ML".
CNNs are the best we have. If they can't deal with adversarial examples, then what can?