Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adversarial image attacks are no joke

unite.ai

Post reply on HN
Print view View original

21–30 of 196 posts

Re: Adversarial image attacks are no joke

#21

Post by josefx »

ravi-delia wrote: post #8

It seems to me theres a difference between attacks that carefully craft an image that slips through the cracks, and an attack that basically exploits the fact that without context, it's hard to figure out what single item is important. If I took a picture of a conch shell on top of my keyboard and sent it to someone, no one would think I was just showing off my keyboard! They'd assume, correctly, that my desk was a m…

The attack also means you can't use a system based on it for content filtering unless you get it to reliably identify multiple objects in a picture. A picture of a conch shell is harmless, a picture of a conch shell and a beheaded person may not be.

Re: Adversarial image attacks are no joke

#22

Post by frazbin »

jowday wrote: post #5
frazbin wrote: post #4

The really scary thing is that this could be used as an excuse to hide production ML models and even the tech used to generate them. Sounds like we can expect the state-of-the-art AI techniques to be jealously guarded eventually. I guess optimism on the ground is enough to have prevented that so far, but once the scales tip away from sharing and towards exploitation.. well, we know it's largely a one-way process on t…

> Sounds like we can expect the state-of-the-art AI techniques to be jealously guarded eventually. This isn’t an eventuality, it’s the current state of the industry.

Hm is that really true? I thought that there was quite a lot of sharing from industry leaders at the research paper and dataset level, and that these could be used imitate production systems given some hacking. Kinda seemed like the majors were enjoying the benefits of the scrutiny afforded to public scientific research, while keeping their monopoly confined to the silicon/speed/throughput axis. Hence all the free AI software toolkits and also high priced specialty hot-off-the-wafer chips you'll never get.

Re: Adversarial image attacks are no joke

#23

Post by siboehm »

mgraczyk wrote: post #14
pixelgeek wrote: post #12

Earlier quoted context omitted.

> For example, consider the case of pasting a sticker on a speed limit sign that causes Teslas to swerve off the road. If your vision system can be caused to swerve off a road by a sticker then maybe it shouldn't be used?

I bet I could cause a significant fraction of human vision systems to get in a crash with a well placed sticker. I'd replace " ".

This has always confused me as well. What would be the reason why some adversary would choose to craft an adversarial example and deploy it in the real world versus the much easier solution to just remove / obscure the sign?

Re: Adversarial image attacks are no joke

#24

Post by dontreact »

If you really wanted to crash cars by altering their visual input, why would you bother with all this complexity? Why not just actually swap the road sign?

Why does the existence of these attacks change the threat landscape at all? If people are already not doing "dumb" attacks like just changing/removing road signs why would they start doing them?

The risk of messing with road signs and throwing off autonomous vehicles really has less to do with adversarial image attacks and more to do with envisioning an impractically brittle system where the decision to stop is based purely on presence/absence of a stop sign and not on a system that has a more general sense of collision-avoidance and situational awareness (like humans do).|

Stepping back more generally, I have still never seen a case where the undetectability of adversarial attacks actually means there is a practical difference to security or safety. If you really think through the impact in the real world, usually the risk is already there: you can just change the input to the image and get bad results, it doesn't affect much that the image is imperceptibly changed. Because the whole point of using an automated vision system is usually that you want to avoid human eyes on the problem.

Re: Adversarial image attacks are no joke

#25

Post by qw501428 »

pixelgeek wrote: post #18
mgraczyk wrote: post #14

Earlier quoted context omitted.

I bet I could cause a significant fraction of human vision systems to get in a crash with a well placed sticker. I'd replace " ".

But that is not a vision issue. That is providing people with incorrect information.

Perhaps a mirror/reflective sticker that blinds drivers near a sharp curve?

Re: Adversarial image attacks are no joke

#26

Post by tonyarkles »

siboehm wrote: post #23
mgraczyk wrote: post #14

Earlier quoted context omitted.

I bet I could cause a significant fraction of human vision systems to get in a crash with a well placed sticker. I'd replace " ".

This has always confused me as well. What would be the reason why some adversary would choose to craft an adversarial example and deploy it in the real world versus the much easier solution to just remove / obscure the sign?

Depending on how big or small it needs to be, potentially for subtlety? Especially on current roads that are shared by humans and self-driving systems, a human observer will immediately notice that something is terribly wrong with a replaced sign.

But... around here at least, signs have stickers or graffiti on them often enough. Like adding the name of a politician under a stop sign: "Stop [Harper]". An appropriately made adversarial example won't stick out visually the same way that a wholesale sign swap will.

Re: Adversarial image attacks are no joke

#27

Post by laura_g »

siboehm wrote: post #23
mgraczyk wrote: post #14

Earlier quoted context omitted.

I bet I could cause a significant fraction of human vision systems to get in a crash with a well placed sticker. I'd replace " ".

This has always confused me as well. What would be the reason why some adversary would choose to craft an adversarial example and deploy it in the real world versus the much easier solution to just remove / obscure the sign?

Because NeurIPS doesn't publish papers on stop sign removal yet :P

Re: Adversarial image attacks are no joke

#28

Post by mrfox321 »

mgraczyk wrote: post #13
voakbasda wrote: post #9

Earlier quoted context omitted.

Your proposed laws do not cut out any exemption for research and experimentation, either with existing systems or potential new ones. This level of regulation would create an impossibly high barrier to entry and ensure that only the established players would remain in the marketplace. The last thing that I want to see is yet more regulatory capture, particularly in an industry that has yet to establish a reasonable b…

None of the things I listed would affect research. Researchers shouldn't be posting these on public highways, and researchers shouldn't be distributing them with the intent to cause harm.

It would once the govt gets involved. It's like saying that weapons research is just a free-for-all. The amount of regulation is correlated with the potential harm to society.

Look at drug research. There is plenty of red tape that hinders it. Although, here, the "harm to society" is defined by the nation state.

However, I agree with your proposals in the top-level comment.

Re: Adversarial image attacks are no joke

#30

Post by Isinlor »

owlbite wrote: post #11
laura_g wrote: post #6

The literature has pretty consistently shown that adversarial examples can be found with only black box access (even with truncated prediction vectors), robustness methods are primarily a cat-and-mouse game between attackers and defenders, and the existence of adversarial examples is likely inevitable ( https://arxiv.org/pdf/1809.02104.pdf ). The big question that remains is - so what? There's exceedingly few use cas…

Self driving cars seem like a dangerous threat vector if an adversarial image can be deployed in such a way as to cause them to commit dangerous maneuvers on demand.

There is plenty of natural "adversarial examples" to worry about.

Like billboard with stop sign on it.

https://youtu.be/-OdOmU58zOw?t=149

Post reply on HN