Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adversarial image attacks are no joke

unite.ai

Post reply on HN
Print view View original

71–80 of 196 posts

Re: Adversarial image attacks are no joke

#71

Post by mgraczyk »

seany wrote: post #61
mgraczyk wrote: post #7

As somebody who works on computer vision, my general take on these things is that adversarial examples are like poison. It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though. It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Inst…

1 and 2 are almost always going to be impossible in the US due to the first amendment (this is a feature not a bug) 3 doesn't seem crazy, but it would practically end up with caps, which might not be what you're looking for 4 This both: seems possible, and will basically never happen due to cost in every little jurisdiction

#1 is certainly not a first amendment violation. In fact, the supreme court still holds that certain restrictions on billboards are allowed even for the purpose of preserving beauty. Safety is a much more compelling interest than beauty, so I don't expect states and cities will lose their ability to regulate road signage.

See Metromedia, Inc. v. San Diego for example.

#2 is expensive and difficult, but that's what we do for explosives, poisons, drugs, etc.

Re: Adversarial image attacks are no joke

#72

Post by 323 »

marcosdumay wrote: post #31
mgraczyk wrote: post #14

Earlier quoted context omitted.

I bet I could cause a significant fraction of human vision systems to get in a crash with a well placed sticker. I'd replace " ".

I'm pretty sure this would fail to kill people on almost every place you could try it. And if it works somewhere, it's because there are other problems with the road that should be fixed. Human driving is full of redundancies, and there is a clear hierarchy of information. People will not rush into a road full of cars going on the other way, it doesn't matter what the signs say. If your automated driving system doesn…

> People will not rush into a road full of cars going on the other way, it doesn't matter what the signs say.

And people would not drive into a river passing through multiple barriers, just because their GPS says so.

https://theweek.com/articles/464674/8-drivers-who-blindly-fo...

https://indianexpress.com/article/trending/bizarre/driver-in...

Re: Adversarial image attacks are no joke

#73

Post by [deleted] »

mgraczyk wrote: post #7

As somebody who works on computer vision, my general take on these things is that adversarial examples are like poison. It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though. It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Inst…

[deleted]

Re: Adversarial image attacks are no joke

#74

Post by ketzo »

mgraczyk wrote: post #7

As somebody who works on computer vision, my general take on these things is that adversarial examples are like poison. It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though. It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Inst…

Relevant XKCD: https://xkcd.com/1958/

> I worry about self-driving car safety features.

> What's to stop someone from painting fake lines on the road, or dropping a cutout of a pedestrian onto a highway, to make cars swerve and crash?

> Except... those things would also work on human drivers. What's stopping people now?

> Yeah, causing car crashes isn't hard.

> I guess it's just that most people aren't murderers?

> Oh, right, I always forget.

> An underappreciated component of our road safety system.

Re: Adversarial image attacks are no joke

#75

Post by JoshuaDavid »

goatlover wrote: post #44
dontreact wrote: post #24

If you really wanted to crash cars by altering their visual input, why would you bother with all this complexity? Why not just actually swap the road sign? Why does the existence of these attacks change the threat landscape at all? If people are already not doing "dumb" attacks like just changing/removing road signs why would they start doing them? The risk of messing with road signs and throwing off autonomous vehic…

> Why not just actually swap the road sign? Because you have to physically do it, as opposed to hacking from anywhere else on the planet. > not on a system that has a more general sense of collision-avoidance and situational awareness (like humans do). Are vision systems to that point yet when it comes to driving vehicles? > Because the whole point of using an automated vision system is usually that you want to avoid…

> Because you have to physically do it, as opposed to hacking from anywhere else on the planet.

My impression is that the adversarial image attacks in question involve physically placing a sticker on something which will be in the view of self-driving cars -- it's not a remote exploit.

Re: Adversarial image attacks are no joke

#77

Post by citilife »

ketzo wrote: post #74
mgraczyk wrote: post #7

As somebody who works on computer vision, my general take on these things is that adversarial examples are like poison. It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though. It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Inst…

Relevant XKCD: https://xkcd.com/1958/ > I worry about self-driving car safety features. > What's to stop someone from painting fake lines on the road, or dropping a cutout of a pedestrian onto a highway, to make cars swerve and crash? > Except... those things would also work on human drivers. What's stopping people now ? > Yeah, causing car crashes isn't hard. > I guess it's just that most people aren't murderers? >…

That's how I feel about most dangerous situations in general and I think the national news highlights one-off events in a way we historically were not used to.

For instance, taking out the United States internet would probably only required 3-4 strategic bombings. I bring this up because Tennessee had one of those bombed Christmas last year -- https://www.theverge.com/2020/12/28/22202822/att-outage-nash...

> This brought down wireless and wired networks across parts of Tennessee, Kentucky, and Alabama

Most people aren't all that concerned about doing damage. Keep people happy and generally you don't have crime.

Re: Adversarial image attacks are no joke

#78

Post by xyzzyz »

aneutron wrote: post #54
mgraczyk wrote: post #7

As somebody who works on computer vision, my general take on these things is that adversarial examples are like poison. It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though. It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Inst…

What you are proposing are what I think would be called a security theater. It gives the illusion of security, but they would absolutely not deter a determined threat actor. The only reason that the water supply isn't poisoned is it's unpractical for a single person to conduct the whole exploit chain: Construct the poison in enough quantities, gain access to facilities supplying the water, and actually throwing the c…

Construct the poison in enough quantities, gain access to facilities supplying the water, and actually throwing the compound in it.

Gaining access is rather easy. You can easily fly drones over most of reservoirs and dump whatever you want into them. Making strong poisons is also relatively easy, eg. dimethylmercury can be easily synthesized by any chemistry graduate.

Re: Adversarial image attacks are no joke

#79

Post by beerandt »

mgraczyk wrote: post #14
pixelgeek wrote: post #12

Earlier quoted context omitted.

> For example, consider the case of pasting a sticker on a speed limit sign that causes Teslas to swerve off the road. If your vision system can be caused to swerve off a road by a sticker then maybe it shouldn't be used?

I bet I could cause a significant fraction of human vision systems to get in a crash with a well placed sticker. I'd replace " ".

There are multiple reasons for signs to have different shapes, sizes, and colors, and this is one of them.

An orange diamond "detour" sign isn't easily confused for a smaller rectangle "one way" sign.

Additionally, there should always be two large "do not enter" plus two large red "wrong way" signs that are visible to a driver from in the intersection before turning.

Something as simple as tape or other coverings on an existing sign should never result in any confusion as to right-of-way for a driver paying attention.

Re: Adversarial image attacks are no joke

#80

Post by seany »

mgraczyk wrote: post #71
seany wrote: post #61

Earlier quoted context omitted.

1 and 2 are almost always going to be impossible in the US due to the first amendment (this is a feature not a bug) 3 doesn't seem crazy, but it would practically end up with caps, which might not be what you're looking for 4 This both: seems possible, and will basically never happen due to cost in every little jurisdiction

#1 is certainly not a first amendment violation. In fact, the supreme court still holds that certain restrictions on billboards are allowed even for the purpose of preserving beauty. Safety is a much more compelling interest than beauty, so I don't expect states and cities will lose their ability to regulate road signage. See Metromedia, Inc. v. San Diego for example. #2 is expensive and difficult, but that's what we…

Someone could stand holding it in protest of self driving cars.
Post reply on HN