Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adversarial image attacks are no joke

unite.ai

Post reply on HN
Print view View original

151–160 of 196 posts

Re: Adversarial image attacks are no joke

#151

Post by brokenmachine »

mgraczyk wrote: post #128
p2p_astroturf wrote: post #127

Earlier quoted context omitted.

Your analysis does not break out of the well known box that is the classical ways of analyzing the security of a computer system (it actually creeps into DRM/TPM territory which is known insecure despite governments with guns). Thus the security of "AI" algorithms remains as insecure as it already was, and should not be used for anything that needs to be secure. If anything, the people who make critical infrastructur…

No, I'm asking you to go to jail if you intentionally try to cause somebody to die.

The people shilling crap AI should bear some responsibility in my opinion.

If your garbage AI powered car can be tricked easier than an 8 year old, who's fault is it?

We wouldn't let an 8 year old drive, but your garbage AI is fine?

Re: Adversarial image attacks are no joke

#152

Post by d110af5ccf »

mgraczyk wrote: post #13
voakbasda wrote: post #9

Earlier quoted context omitted.

Your proposed laws do not cut out any exemption for research and experimentation, either with existing systems or potential new ones. This level of regulation would create an impossibly high barrier to entry and ensure that only the established players would remain in the marketplace. The last thing that I want to see is yet more regulatory capture, particularly in an industry that has yet to establish a reasonable b…

None of the things I listed would affect research. Researchers shouldn't be posting these on public highways, and researchers shouldn't be distributing them with the intent to cause harm.

This is naive.

> researchers shouldn't be distributing them with the intent to cause harm

It could be used to cause harm and publishing your work is distributing it (by definition). Similar laws have been (and are) used to target people for unjust reasons.

Defacing street signs is already illegal.

Making various abstract drawings illegal to wear on your clothing (or print on a sticker) is a terrible idea. If the algorithm used by a product can't handle a pedestrian wearing a sweatshirt with an adversarial example on it then that product simply isn't suitable for public use.

Re: Adversarial image attacks are no joke

#153

Post by brokenmachine »

boomboomsubban wrote: post #116
KingMachiavelli wrote: post #112

Earlier quoted context omitted.

No those are completely different. The physical act of owning an explosive can be made illegal and is . In the US the act of owning and expressing an element of speech is protected under US law. You are getting close to something with you second statement. There are laws that criminalize actions like yelling 'Fire' inside a movie theater or provoking a fight (fighting words). Essentially these laws isolate the protec…

> The physical act of owning an explosive can be made illegal and is. Then let me change my example to show legal items being used with the intent to cause harm is still illegal. I'm free to put razors into candy, but if I hand it out on Halloween it'd be illegal. >However, it would be an extreme stretch to apply or expand these to apply to simply wearing a t-shirt. There is already plenty of case law that says weari…

>wearing it in public is endangering the public.

Only because you're driving a car that was programmed by monkeys and sold by PT Barnum.

If your car can't tell the difference between a street sign and a T-shirt, it's really not fully self driving, is it?

Re: Adversarial image attacks are no joke

#154

Post by sieabahlpark »

tshaddox wrote: post #138
GhettoComputers wrote: post #135

Earlier quoted context omitted.

Looks like they were innocent. https://www.law.umich.edu/special/exoneration/Pages/casedeta... > In 1998, the defense filed a post-conviction motion for a new trial. At a hearing on the motion, several witnesses testified that they had driven by the intersection days before the accident and the stop sign was already down. Some of the witnesses said that after the charges were filed, they reported to both Hillsborough…

Fair enough, I'm not very familiar with the details of that case, but my overall points are that 1) it's easy to remove traffic signs (and thoughtless kids probably do it every so often as a dumb joke), 2) this can definitely lead to bad accidents (with human drivers), and 3) the legal system will likely at least consider whether the people who tampered with the traffic signs are culpable for the accidents.

Yet you found it critical enough to write a comment using it as the basis for your argument...?

Re: Adversarial image attacks are no joke

#155

Post by littlestymaar »

ineedasername wrote: post #149
littlestymaar wrote: post #136

Earlier quoted context omitted.

A few people taking completely stupid decisions is always going to happen. There's also drunk people or elderly people with poor vision who end up in the wrong direction on the highway. Even if it happens every day on a global scale, it's an incredibly small fraction of drivers. A targeted attack fooling current-technology IA could lead hundreds of cars in the wrong direction at the same time in a single place. It's…

Hundreds you say? Think again, it's happened. (Well, 100) https://www.google.com/amp/s/www.cbc.ca/amp/1.5192656

That's a fun trivia, but that's not really relevant to the discussion here: those people took no risk for their lives when following these instructions. Making hundreds of people go in the wrong place is easy, but the difference between a bad IA and a human driver is that the majority of drivers realize when they're doing something dangerous and stop.

This common sense of danger is what IAs must have before we can trust them on the streets.

For instance in the Uber fatal crash: the IA correctly labelled the cyclist as a cyclist several times then revised its judgment, then definetely labelled it as a cyclist but “oops too late”. A human would be like “wait, I think I saw a cyclist, slow down and figure out what this really was”. Obviously with the current computer vision technology you cannot do that, otherwise your car would be afraid of everything and refuse to even start, because there's way too much mislabelling happening at some point or another.

Re: Adversarial image attacks are no joke

#156

Post by mannerheim »

mgraczyk wrote: post #140
mannerheim wrote: post #131

Earlier quoted context omitted.

> 2. People who create and distribute these stickers knowing their purpose should go to prison. This would surely not pass constitutional muster.

You might be surprised to learn that the US government regulates things that are much closer to what you consider "speech". For example, > Federal law prohibits the possession with intent to sell or distribute obscenity, to send, ship, or receive obscenity, to import obscenity, and to transport obscenity across state borders for purposes of distribution. https://www.justice.gov/criminal-ceos/citizens-guide-us-fede...…

Is federal obscenity law enforced? I doubt it can be enforced either, with the decision in Reno v. ACLU that struck down similar provisions of the Communications Decency Act as violating the First Amendment.

> If it's illegal to posses something, the government can ban offering to sell and distribute it.

This is begging the question, because first you would have to show that banning the mere possession of adversarial images doesn't violate the First Amendment. Otherwise you can make it illegal to possess books in a way to prevent the sale and distribution of literature.

Re: Adversarial image attacks are no joke

#157

Post by d110af5ccf »

mgraczyk wrote: post #140
mannerheim wrote: post #131

Earlier quoted context omitted.

> 2. People who create and distribute these stickers knowing their purpose should go to prison. This would surely not pass constitutional muster.

You might be surprised to learn that the US government regulates things that are much closer to what you consider "speech". For example, > Federal law prohibits the possession with intent to sell or distribute obscenity, to send, ship, or receive obscenity, to import obscenity, and to transport obscenity across state borders for purposes of distribution. https://www.justice.gov/criminal-ceos/citizens-guide-us-fede...…

To be fair, obscenity laws don't seem (to me) to be in keeping with the rest of the laws surrounding freedom of speech in the US. They are shockingly vague and have been used to prosecute people for artwork (!!!) in the past. Court verdicts for such laws seem to be mixed - sometimes they go along with them, sometimes they side with freedom of expression.

> offers to engage in illegal transactions

Just to clarify, that means "offers to engage in crime". Other than the aforementioned obscenity laws freedom of expression is generally quite well protected in the US so there won't be a crime in the first place (and thus related transactions won't be illegal).

> If it's illegal to posses something, the government can ban offering to sell and distribute it.

I think you misunderstand slightly. If it's illegal to possess something then it is _already_ (to the best of my knowledge) illegal to offer to sell or distribute it. The question is what the government is and isn't allowed to ban possession of.

Interestingly, in the case of "obscene" materials possession itself isn't banned. Only import, sale, and distribution.

The issue with "People who create and distribute these stickers knowing their purpose should go to prison." is that such a wording seemingly bans them outright regardless of intent. That is an affront to freedom of expression. If I want to craft adversarial examples I shouldn't need to justify my intentions and seek permission up front. An action should need to be justified as illegal on a case by case basis, not the other way around.

Note that it is already illegal to deface street signs so by extension if you create and distribute stickers with the express intention that they be used that way then presumably you are already violating the law today. On the other hand, such stickers are not (currently or ever, I hope) inherently illegal in and of themselves.

Re: Adversarial image attacks are no joke

#158

Post by ineedasername »

littlestymaar wrote: post #155
ineedasername wrote: post #149

Earlier quoted context omitted.

Hundreds you say? Think again, it's happened. (Well, 100) https://www.google.com/amp/s/www.cbc.ca/amp/1.5192656

That's a fun trivia, but that's not really relevant to the discussion here: those people took no risk for their lives when following these instructions. Making hundreds of people go in the wrong place is easy, but the difference between a bad IA and a human driver is that the majority of drivers realize when they're doing something dangerous and stop. This common sense of danger is what IAs must have before we can tr…

I just meant it as an example where a large group of humans will follow what a computer tells them even when it is pretty clearly wrong. It's not hard to conceive of possibilities where the danger wouldn't be obvious until after person blindly follows the instructions.

Re: Adversarial image attacks are no joke

#159

Post by d110af5ccf »

mannerheim wrote: post #156
mgraczyk wrote: post #140

Earlier quoted context omitted.

You might be surprised to learn that the US government regulates things that are much closer to what you consider "speech". For example, > Federal law prohibits the possession with intent to sell or distribute obscenity, to send, ship, or receive obscenity, to import obscenity, and to transport obscenity across state borders for purposes of distribution. https://www.justice.gov/criminal-ceos/citizens-guide-us-fede...…

Is federal obscenity law enforced? I doubt it can be enforced either, with the decision in Reno v. ACLU that struck down similar provisions of the Communications Decency Act as violating the First Amendment. > If it's illegal to posses something, the government can ban offering to sell and distribute it. This is begging the question, because first you would have to show that banning the mere possession of adversarial…

> Is federal obscenity law enforced?

It seems to vary. Typically not, and when attempts are made court verdicts seem to be mixed. But unfortunately sometimes people do get convicted and the convictions upheld. For example, United States v. Whorley. https://caselaw.findlaw.com/us-4th-circuit/1431669.html

Re: Adversarial image attacks are no joke

#160

Post by mgraczyk »

d110af5ccf wrote: post #157
mgraczyk wrote: post #140

Earlier quoted context omitted.

You might be surprised to learn that the US government regulates things that are much closer to what you consider "speech". For example, > Federal law prohibits the possession with intent to sell or distribute obscenity, to send, ship, or receive obscenity, to import obscenity, and to transport obscenity across state borders for purposes of distribution. https://www.justice.gov/criminal-ceos/citizens-guide-us-fede...…

To be fair, obscenity laws don't seem (to me) to be in keeping with the rest of the laws surrounding freedom of speech in the US. They are shockingly vague and have been used to prosecute people for artwork (!!!) in the past. Court verdicts for such laws seem to be mixed - sometimes they go along with them, sometimes they side with freedom of expression. > offers to engage in illegal transactions Just to clarify, tha…

I'm not misunderstanding, although I agree that all this is largely beside the point.

It's not obviously true that the illegality of possession automatically implies the illegality of the offer to sell. Consider for example a case where the seller doesn't actually possess the item, but merely offers to sell it. In this case, the supreme court upheld a law making such offers illegal. Without the ruling, you might argue that the mere offer is protected speech.

The point of all this is really indirect though. I'm just saying that in cases where "speech" concerns illegal items, like nuclear weapons, child pornography, and drugs, sometimes the government is given more leeway in controlling speech related to items.

Post reply on HN