Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adversarial image attacks are no joke

unite.ai

Post reply on HN
Print view View original

101–110 of 196 posts

Re: Adversarial image attacks are no joke

#101

Post by NoGravitas »

OldHand2018 wrote: post #40

I see a completely different attack vector here. Lawyers. If you are selling a product or service that has been trained on a dataset that contains copyrighted photos you don't have permission to use and I can "prove it" enough to get you into court and into the discovery phase, you are screwed. I'll get an injunction that shuts you down while we talk about how much money you have to pay me. And lol, if any of those p…

I dunno, Microsoft seem to think they can get away with training autocomplete on copyrighted source code that they don't have permission to use.

Re: Adversarial image attacks are no joke

#102

Post by glitchc »

mgraczyk wrote: post #7

As somebody who works on computer vision, my general take on these things is that adversarial examples are like poison. It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though. It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Inst…

Adversarial examples don't confuse people, only algorithms.

Perhaps you need to face the fact that if the CV algorithm fails against these examples when humans don't, then the CV algorithm is too brittle and should not be used in the real world. I don't trust my life to your "It kinda looks like a road, oh wait it's a pylon, I've been tricked, BAM!" dumpster fire of an algorithm.

We used to have to craft robustness into algorithms based on the false positive rate. Nobody looks at a CFAR style approach anymore, and it shows. The state of the art approach of pinning everything on ML is a dead-end for CV.

Re: Adversarial image attacks are no joke

#103

Post by aneutron »

eximius wrote: post #100
p_j_w wrote: post #89

Earlier quoted context omitted.

>It doesn't mean we shouldn't put deterrents. GP doesn't say we shouldn't, but rather that it's not good enough.

Generally calling something security theatre has an implication that it shouldnt be done because of its inefficacy and the availability of robust alternatives (e.g., port knocking is theatre when we can have robust security on known ports with minimal configuration and cryptography).

While I do agree that security theater does have a connotation for things that have no reason to be done, I only meant that it's not enough. It's theater in the sense that it would only provide a sense of safety, not solve the actual underlying issue or vulnerability class.

Re: Adversarial image attacks are no joke

#104

Post by boomboomsubban »

nemo44x wrote: post #98
mgraczyk wrote: post #71

Earlier quoted context omitted.

#1 is certainly not a first amendment violation. In fact, the supreme court still holds that certain restrictions on billboards are allowed even for the purpose of preserving beauty. Safety is a much more compelling interest than beauty, so I don't expect states and cities will lose their ability to regulate road signage. See Metromedia, Inc. v. San Diego for example. #2 is expensive and difficult, but that's what we…

Explosives, poisons, drugs aren’t speech. Printing the chemistry for then is protected speech. If I wanted to print an image and put it on a t-shirt that would trick a computer driven car into doing something if its cameras saw my shirt, that’s not my problem. The barrier to entry is much lower too so I think it’s up to the engineers to solve it instead of trying to dump the hard problems on society.

This is like saying "if I set up a movement based explosive in a public place, and you just happened to walk by it, that's not my problem." Yes it is, you took actions that you knew could severely harm people.

Re: Adversarial image attacks are no joke

#105

Post by glitchc »

mgraczyk wrote: post #14
pixelgeek wrote: post #12

Earlier quoted context omitted.

> For example, consider the case of pasting a sticker on a speed limit sign that causes Teslas to swerve off the road. If your vision system can be caused to swerve off a road by a sticker then maybe it shouldn't be used?

I bet I could cause a significant fraction of human vision systems to get in a crash with a well placed sticker. I'd replace " ".

I bet you can't. Humans are anti-fragile and can compensate with other knowledge.

Re: Adversarial image attacks are no joke

#106

Post by rStar »

mgraczyk wrote: post #7

As somebody who works on computer vision, my general take on these things is that adversarial examples are like poison. It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though. It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Inst…

> 1. People who post these stickers should go to prison. 2. People who create and distribute these stickers knowing their purpose should go to prison. 3. Tesla should be civilly liable for cases where preventing such an incident was possible with known technology. 4. Roads should be modified over time to make it more difficult to do this attack.

Translation: everyone else in the universe is responsible for solving my problem, and also I am not responsible for solving my problem, but i do want to profit from the current state of everything being broken all the time, and, i tell my family to keep their hands on the wheel

Re: Adversarial image attacks are no joke

#107

Post by ClumsyPilot »

b3morales wrote: post #45
pixelgeek wrote: post #18

Earlier quoted context omitted.

But that is not a vision issue. That is providing people with incorrect information.

The specific trick doesn't really matter; the point is that it's possible to maliciously create a situation that makes human pilots act dangerously. We accept that the possibility can't be made nil, and we have post facto rules to deal with it. The same principle applies to traps for machines.

Nope, it 0.1% of humans crash but 100% of teslas crash that's not 'the same'

Re: Adversarial image attacks are no joke

#108

Post by darepublic »

mgraczyk wrote: post #7

As somebody who works on computer vision, my general take on these things is that adversarial examples are like poison. It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though. It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Inst…

So your solution is to create a totalitarian state. So your flaky software can be secure. No thanks

Re: Adversarial image attacks are no joke

#109

Post by donkarma »

dhosek wrote: post #32

Whenever these discussions come up, I often think of a time I was driving on a two-lane road in rural Ohio in the 90s and at one point the center stripe curved into the lane (presumably because the driver of the striping truck pulled over without turning off the striper) and I started to curve off the road thanks to subconscious interpretation of the cue. I caught myself before I drove into a corn field, but human vi…

You caught yourself, didn't you?

Re: Adversarial image attacks are no joke

#110

Post by lodovic »

Laremere wrote: post #48
voakbasda wrote: post #9

Earlier quoted context omitted.

Your proposed laws do not cut out any exemption for research and experimentation, either with existing systems or potential new ones. This level of regulation would create an impossibly high barrier to entry and ensure that only the established players would remain in the marketplace. The last thing that I want to see is yet more regulatory capture, particularly in an industry that has yet to establish a reasonable b…

INAL, but actually putting adversarial image attacks on real roads is already illegal. If you modify a street sign, and as a result someone dies, that's a fairly easy case of Involuntary manslaughter. At a minimum, you can't modify street signs. Eg in Washington State: RCW 47.36.130 Meddling with signs prohibited. No person shall without lawful authority attempt to or in fact _alter_, deface, injure, knock down, or r…

Looks like it will be illegal to wear that shirt with the Obama flower image if there is an AI face recognition system installed over the highway though.
Post reply on HN