Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adversarial image attacks are no joke

unite.ai

Post reply on HN
Print view View original

61–70 of 196 posts

Re: Adversarial image attacks are no joke

#61

Post by seany »

mgraczyk wrote: post #7

As somebody who works on computer vision, my general take on these things is that adversarial examples are like poison. It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though. It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Inst…

1 and 2 are almost always going to be impossible in the US due to the first amendment (this is a feature not a bug)

3 doesn't seem crazy, but it would practically end up with caps, which might not be what you're looking for

4 This both: seems possible, and will basically never happen due to cost in every little jurisdiction

Re: Adversarial image attacks are no joke

#63

Post by eximius »

seany wrote: post #61
mgraczyk wrote: post #7

As somebody who works on computer vision, my general take on these things is that adversarial examples are like poison. It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though. It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Inst…

1 and 2 are almost always going to be impossible in the US due to the first amendment (this is a feature not a bug) 3 doesn't seem crazy, but it would practically end up with caps, which might not be what you're looking for 4 This both: seems possible, and will basically never happen due to cost in every little jurisdiction

Defacing property is not free speech...?

Re: Adversarial image attacks are no joke

#64

Post by eximius »

aneutron wrote: post #54
mgraczyk wrote: post #7

As somebody who works on computer vision, my general take on these things is that adversarial examples are like poison. It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though. It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Inst…

What you are proposing are what I think would be called a security theater. It gives the illusion of security, but they would absolutely not deter a determined threat actor. The only reason that the water supply isn't poisoned is it's unpractical for a single person to conduct the whole exploit chain: Construct the poison in enough quantities, gain access to facilities supplying the water, and actually throwing the c…

> It gives the illusion of security, but they would absolutely not deter a determined threat actor.

Sure. And the threat of jail/imprisonment doesn't deter determined murderer's. It doesn't mean we shouldn't put deterrents.

Re: Adversarial image attacks are no joke

#65

Post by hacoo »

seany wrote: post #61
mgraczyk wrote: post #7

As somebody who works on computer vision, my general take on these things is that adversarial examples are like poison. It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though. It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Inst…

1 and 2 are almost always going to be impossible in the US due to the first amendment (this is a feature not a bug) 3 doesn't seem crazy, but it would practically end up with caps, which might not be what you're looking for 4 This both: seems possible, and will basically never happen due to cost in every little jurisdiction

I doubt 1 would be protected by the first amendment. It’s arguably equivalent to spraying graffiti on a stop sign so it’s unrecognizable.

It would be an extremely difficult to enforce though.

Re: Adversarial image attacks are no joke

#66

Post by AnimalMuppet »

inetknght wrote: post #56
marcosdumay wrote: post #31

Earlier quoted context omitted.

I'm pretty sure this would fail to kill people on almost every place you could try it. And if it works somewhere, it's because there are other problems with the road that should be fixed. Human driving is full of redundancies, and there is a clear hierarchy of information. People will not rush into a road full of cars going on the other way, it doesn't matter what the signs say. If your automated driving system doesn…

> People will not rush into a road full of cars going on the other way, it doesn't matter what the signs say. You might want to watch the one-way roads in big cities. It happens a lot more often than you assume. It also is (usually) self-correcting: oncoming traffic will honk, stop, or move around. The offender will (usually) realize their mistake and try to correct. Sometimes, though, that's not enough. Searching "k…

Been there, done that (except the "killed" part). It was in a heavy fog. I was doing well to find a street at all, and it turned out to be one way the wrong way (the only such street in town). I figured it out when I saw wall-to-wall headlights coming at me out of the fog, and made a fast move for the curb...

So, yeah. People react. Which brings up the question: How well do self-driving AIs respond to a wrong-way driver? How well do self-driving AIs recover when they are the wrong-way driver, and they suddenly have enough data to realize that?

Re: Adversarial image attacks are no joke

#67

Post by AnimalMuppet »

bryanrasmussen wrote: post #41
marcosdumay wrote: post #31

Earlier quoted context omitted.

I'm pretty sure this would fail to kill people on almost every place you could try it. And if it works somewhere, it's because there are other problems with the road that should be fixed. Human driving is full of redundancies, and there is a clear hierarchy of information. People will not rush into a road full of cars going on the other way, it doesn't matter what the signs say. If your automated driving system doesn…

the claim is not that automated driving systems are ready for use, the claim is that if you do things in order to compromise a system that has a good chance of killing people and then does kill people that should be illegal, which of course it already is.

Yeah. "Voluntary manslaughter" and "malicious mischief" are already things you can prosecute for.

Re: Adversarial image attacks are no joke

#68

Post by version_five »

mgraczyk wrote: post #7

As somebody who works on computer vision, my general take on these things is that adversarial examples are like poison. It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though. It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Inst…

I lot of this has been touched on already, but I think your rules could be reframed a bit to try simplify lawmaking and avoid security theatre as was mentioned.

First, I assume it's already illegal to be "adversarial" to drivers. A bright light or changing signs etc already do that now. For example look at all the laser pointer stuff with planes.

Second, I don't think self driving cars are just using the softmax output of an object detector as a direct input to car control decisions. In the absence of a stop sign, the expected behavior would be common sense and caution, the same as if someone removed the sign. If the SDC logic is not robust in this way, it's not safe for many other reasons.

With this in mind, I think the situation is probably already reasonable well covered in existing regulations.

Re: Adversarial image attacks are no joke

#69

Post by boomboomsubban »

hacoo wrote: post #65
seany wrote: post #61

Earlier quoted context omitted.

1 and 2 are almost always going to be impossible in the US due to the first amendment (this is a feature not a bug) 3 doesn't seem crazy, but it would practically end up with caps, which might not be what you're looking for 4 This both: seems possible, and will basically never happen due to cost in every little jurisdiction

I doubt 1 would be protected by the first amendment. It’s arguably equivalent to spraying graffiti on a stop sign so it’s unrecognizable. It would be an extremely difficult to enforce though.

Graffiti would just cause people to drive unsafely, in that hypothetical the sticker directly causes crashes. It'd be something like attempted murder.

Re: Adversarial image attacks are no joke

#70

Post by genewitch »

Isinlor wrote: post #30
owlbite wrote: post #11

Earlier quoted context omitted.

Self driving cars seem like a dangerous threat vector if an adversarial image can be deployed in such a way as to cause them to commit dangerous maneuvers on demand.

There is plenty of natural "adversarial examples" to worry about. Like billboard with stop sign on it. https://youtu.be/-OdOmU58zOw?t=149

I'll be more inclined to start believing that self driving / autonomous vehicles are actually "coming soon" when the federal government decrees it is illegal to wear clothing with certain markings/colors. No red octogons, no reflective red and white parts, no yellow vertical stripes, etc.

I don't think that "cause an air to fail to stop" is the correct threat to address, I think "making AI stop and therefore cause traffic" is.

Wake me up when I can have any two arbitrary addresses as start and end points and a machine or computer can drive me between them, 24/7/365 - barring road closures or whatever.

Post reply on HN