Adversarial Learning for Good: On Deep Learning Blindspots • Hacker News Board

Re: Adversarial Learning for Good: On Deep Learning Blindspots

#11

Post by daenz » Sat, Dec 30, 2017, 5:01 AM UTC

doener wrote: post #7

daenz wrote: post #5

The video example of a turtle being misclassified as a rifle is pretty scary https://www.youtube.com/watch?v=piYnd_wYlT8 We're approaching a world where AI will be more and more relied upon in dangerous situations. Imagine someone getting killed for something ridiculous like inadvertently holding an adversarial example. Public trust would have a hard time recovering.

>Imagine someone getting killed for something ridiculous like inadvertently holding an adversarial example. This would never be the case with humans, right? No.

With humans there is accountability (in theory). Who is accountable for the model?

Re: Adversarial Learning for Good: On Deep Learning Blindspots

#12

Post by beefield » Sat, Dec 30, 2017, 8:25 AM UTC

marcosdumay wrote: post #9

beefield wrote: post #6

I wonder how sensitive these results are. I mean, if you run one more learning round of the network, does the rifle turn to turtle in the eyes of the network immediately so that you would need to generate a new turtle for every single network? Or does that turtle look like rifle for all current neural networks? Or, most likely, somewhere in between?

If you don't change the data, why would the network change its results with just some more learning iterations? It's already converging.

Because if I understtod anything correctly, the methods tries to find the smallest possible changes that causes the network to make the incorrect classofication. And these smallest possible changes just might be very sensitive to whatever small random ripple weight changes a network has. But I am far from neural network expert, so I can't really answer.

Re: Adversarial Learning for Good: On Deep Learning Blindspots

#13

Post by trhway » Sat, Dec 30, 2017, 8:38 AM UTC

daenz wrote: post #11

doener wrote: post #7

Earlier quoted context omitted.

>Imagine someone getting killed for something ridiculous like inadvertently holding an adversarial example. This would never be the case with humans, right? No.

With humans there is accountability (in theory). Who is accountable for the model?

model is adjustable/correctable/trainable/improvable. Humans are much less so.