Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adversarial image attacks are no joke

unite.ai

Post reply on HN
Print view View original

141–150 of 196 posts

Re: Adversarial image attacks are no joke

#141

Post by Eelongate »

hermitdev wrote: post #121
catlikesshrimp wrote: post #88

Earlier quoted context omitted.

I will go further and say that almost everytime there is an accident the driver is somehow impaired. Lack of sleep, drugs, illness (old age included, mental disease), poor judgement (young age included, emotional distress) Humans are surprisingly good at driving under normal conditions.

Some are. Some are not. Last week, I was nearly in two accidents on maybe a 1 mile trip to the store from my house. Both times were people pulling out of traffic, ignoring right of way. I prevented the accidents that would have resulted from these two separate idiots. I have also been in over 20 accidents in my 25 years of driving, the vast majority of those having been rear-ended and none were my fault. In my experi…

> I have also been in over 20 accidents in my 25 years of driving, the vast majority of those having been rear-ended and none were my fault.

That's way too many to take your word for it. Where there's smoke, there's fire. Well, maybe not... but you sure as shit should suspect a fire.

Re: Adversarial image attacks are no joke

#142

Post by ineedasername »

gh0std3v wrote: post #118
aneutron wrote: post #54

Earlier quoted context omitted.

What you are proposing are what I think would be called a security theater. It gives the illusion of security, but they would absolutely not deter a determined threat actor. The only reason that the water supply isn't poisoned is it's unpractical for a single person to conduct the whole exploit chain: Construct the poison in enough quantities, gain access to facilities supplying the water, and actually throwing the c…

> What you are proposing are what I think would be called a security theater. I don't think putting people to prison for, say, flipping a Tesla by screwing with its computer vision algorithm is security theatre. Rather, it's accountability. I'm pretty sure most people are aware that you cannot stop a determined attacker from breaking a system (which is exactly why Spectre mitigations were implemented as soon as the v…

The theater is in the (somewhat) illusory notion that precautions could prevent it from happening. Prosecuting a crime is absolutely not the same thing as actual security. If a modestly funded department at a university can do this, it's within reach for pretty much any state-level actor. And just like deepfakes are much easier & available for scammers today than they were 5 years ago, the same will go for adversarial images.

5 years ago it would have been pretty much unthinkable that a ransomware attack could actually take down most of the eastern US petrol pipeline infrastructure but here we are, no one prosecuted, and apparently the only thing stopping other high profile attacks is the forebearance and self-policing of the thieves themselves.

Re: Adversarial image attacks are no joke

#143

Post by MauranKilom »

ClumsyPilot wrote: post #107
b3morales wrote: post #45

Earlier quoted context omitted.

The specific trick doesn't really matter; the point is that it's possible to maliciously create a situation that makes human pilots act dangerously. We accept that the possibility can't be made nil, and we have post facto rules to deal with it. The same principle applies to traps for machines.

Nope, it 0.1% of humans crash but 100% of teslas crash that's not 'the same'

So would it be better/fine if Tesla randomized the CV ML models to some extent? I actually feel the answer is probably "yes", surprisingly...

Re: Adversarial image attacks are no joke

#144

Post by ineedasername »

eximius wrote: post #100
p_j_w wrote: post #89

Earlier quoted context omitted.

>It doesn't mean we shouldn't put deterrents. GP doesn't say we shouldn't, but rather that it's not good enough.

Generally calling something security theatre has an implication that it shouldnt be done because of its inefficacy and the availability of robust alternatives (e.g., port knocking is theatre when we can have robust security on known ports with minimal configuration and cryptography).

It's also theater if the ratio between actual protection and perceived protection is highly disproportionate, like with the TSA in the US.

Re: Adversarial image attacks are no joke

#145

Post by [deleted] »

aneutron wrote: post #54
mgraczyk wrote: post #7

As somebody who works on computer vision, my general take on these things is that adversarial examples are like poison. It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though. It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Inst…

What you are proposing are what I think would be called a security theater. It gives the illusion of security, but they would absolutely not deter a determined threat actor. The only reason that the water supply isn't poisoned is it's unpractical for a single person to conduct the whole exploit chain: Construct the poison in enough quantities, gain access to facilities supplying the water, and actually throwing the c…

[deleted]

Re: Adversarial image attacks are no joke

#146

Post by d110af5ccf »

Vetch wrote: post #53
Ozzie_osman wrote: post #38

Earlier quoted context omitted.

You put too much faith in humans. Things like stop sign removal have caused deaths in the past. https://www.nytimes.com/1997/06/21/us/3-are-sentenced-to-15-...

Humans are not so bad as drivers. Your example is an event from over 2 decades ago and was deemed newsworthy. Humans drive in all kinds of conditions but death rate is about 1 per 100 million miles driven. A search reveals crashes to be on the order of hundreds of collisions per 100 million miles driven. Age, country, intoxication level, road design and laws, road and environmental conditions also play a major role s…

> a country where road laws are merely friendly suggestions (and considering the chaos of driving in those countries, the rates are actually surprisingly low)

I wonder how well self driving algorithms would compare if tested in such a "hostile" environment? Perhaps we shouldn't allow them on more "friendly" roads until they can consistently surpass human performance under such adverse conditions.

Re: Adversarial image attacks are no joke

#147

Post by ineedasername »

glitchc wrote: post #102
mgraczyk wrote: post #7

As somebody who works on computer vision, my general take on these things is that adversarial examples are like poison. It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though. It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Inst…

Adversarial examples don't confuse people, only algorithms. Perhaps you need to face the fact that if the CV algorithm fails against these examples when humans don't, then the CV algorithm is too brittle and should not be used in the real world. I don't trust my life to your "It kinda looks like a road, oh wait it's a pylon, I've been tricked, BAM!" dumpster fire of an algorithm. We used to have to craft robustness i…

Optical illusions are a sort of adversarial approach for humans, but we're advanced enough that it would be a bit harder to weaponize. In a course I've taught on propaganda though, about 30-50% of the class always fails my example on implanting false memories & "dog whistling"

Humans are absolutely susceptible to a wide variety of adversarial attacks.

Re: Adversarial image attacks are no joke

#148

Post by nerdponx »

ineedasername wrote: post #142
gh0std3v wrote: post #118

Earlier quoted context omitted.

> What you are proposing are what I think would be called a security theater. I don't think putting people to prison for, say, flipping a Tesla by screwing with its computer vision algorithm is security theatre. Rather, it's accountability. I'm pretty sure most people are aware that you cannot stop a determined attacker from breaking a system (which is exactly why Spectre mitigations were implemented as soon as the v…

The theater is in the (somewhat) illusory notion that precautions could prevent it from happening. Prosecuting a crime is absolutely not the same thing as actual security. If a modestly funded department at a university can do this, it's within reach for pretty much any state-level actor. And just like deepfakes are much easier & available for scammers today than they were 5 years ago, the same will go for adversaria…

Making it very expensive to do a thing still reduces the chance of someone doing the thing. How many more murders of passion would happen if murder wasn't illegal?

Laws against murder don't prevent murder from ever happening, but they ensure that committing it is weighed against very high costs.

Perhaps there are other ways to reduce the chance of bad things happening, like reducing opportunities for the bad thing to happen in the first place (eg. not overly relying on computer vision).

Re: Adversarial image attacks are no joke

#149

Post by ineedasername »

littlestymaar wrote: post #136
323 wrote: post #72

Earlier quoted context omitted.

> People will not rush into a road full of cars going on the other way, it doesn't matter what the signs say. And people would not drive into a river passing through multiple barriers, just because their GPS says so. https://theweek.com/articles/464674/8-drivers-who-blindly-fo... https://indianexpress.com/article/trending/bizarre/driver-in...

A few people taking completely stupid decisions is always going to happen. There's also drunk people or elderly people with poor vision who end up in the wrong direction on the highway. Even if it happens every day on a global scale, it's an incredibly small fraction of drivers. A targeted attack fooling current-technology IA could lead hundreds of cars in the wrong direction at the same time in a single place. It's…

Hundreds you say? Think again, it's happened. (Well, 100)

https://www.google.com/amp/s/www.cbc.ca/amp/1.5192656

Re: Adversarial image attacks are no joke

#150

Post by nerdponx »

genewitch wrote: post #70
Isinlor wrote: post #30

Earlier quoted context omitted.

There is plenty of natural "adversarial examples" to worry about. Like billboard with stop sign on it. https://youtu.be/-OdOmU58zOw?t=149

I'll be more inclined to start believing that self driving / autonomous vehicles are actually "coming soon" when the federal government decrees it is illegal to wear clothing with certain markings/colors. No red octogons, no reflective red and white parts, no yellow vertical stripes, etc. I don't think that "cause an air to fail to stop" is the correct threat to address, I think "making AI stop and therefore cause tr…

If I really had to choose, would rather have freedom of expression than AI cars. But maybe that's just me.
Post reply on HN