Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adversarial image attacks are no joke

unite.ai

Post reply on HN
Print view View original

131–140 of 196 posts

Re: Adversarial image attacks are no joke

#131

Post by mannerheim »

mgraczyk wrote: post #7

As somebody who works on computer vision, my general take on these things is that adversarial examples are like poison. It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though. It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Inst…

> 2. People who create and distribute these stickers knowing their purpose should go to prison.

This would surely not pass constitutional muster.

Re: Adversarial image attacks are no joke

#132

Post by karaterobot »

I don't like the idea that self-driving cars were in use, and suddenly became compromised, but only because people could die as a result. Other than that, I can't think of a situation where I would be upset if computer vision was considered too unreliable to be misused by governments or abused by advertisers, which is how it'll go otherwise.

Re: Adversarial image attacks are no joke

#133

Post by OldHand2018 »

KingMachiavelli wrote: post #113
OldHand2018 wrote: post #40

I see a completely different attack vector here. Lawyers. If you are selling a product or service that has been trained on a dataset that contains copyrighted photos you don't have permission to use and I can "prove it" enough to get you into court and into the discovery phase, you are screwed. I'll get an injunction that shuts you down while we talk about how much money you have to pay me. And lol, if any of those p…

IIRC simply training on copyrighted material is completely fine or at least you can claim fair use. As long as the market of the copyrighted material is not 'AI data training set' then it should be OK. Essentially scraping images from the internet is OK but using a pirated copyrighted commercial AI data training set is not. (Fair use doesn't necessarily exclude use for a commercial/sold product.) But if the AI model…

The article contains a link to an Adobe blog that talks about fair use of copyrighted material.

It references the fair use doctrine in a way that is not fully analogous to this type of use and mentions the Google books case. It also mentions that this is not settled law. It's clear that the author wants it to be fair use, but that might cloud their analysis.

Keep in mind that Google was scanning books that the legitimate owner of the physical books gave them permission to scan. If I buy a book and want to use it to train my model, fair use says I am free to do so. If I grab an unauthorized torrent of a training set, itself containing images were not legitimately purchased or licensed, there is absolutely no case law that I know of that says it is ok. I have to spend my money on lawyers trying to argue that I'm in the clear with no guarantee of success.

Maybe I'm wrong - I'd love to hear a convincing argument to the contrary!

Re: Adversarial image attacks are no joke

#134

Post by flerovium »

mgraczyk wrote: post #58
aneutron wrote: post #54

Earlier quoted context omitted.

What you are proposing are what I think would be called a security theater. It gives the illusion of security, but they would absolutely not deter a determined threat actor. The only reason that the water supply isn't poisoned is it's unpractical for a single person to conduct the whole exploit chain: Construct the poison in enough quantities, gain access to facilities supplying the water, and actually throwing the c…

Right, and the protections for poison are also security theater for the same reason. In the real world that's ok. > The only reason that the water supply isn't poisoned is it's unpractical for a single person to conduct the whole exploit chain It's a quantitative question, just like with computer vision. If you don't like the poison example, consider viral DNA, which is also dangerous in the right hands and does not…

The lab leak theory isn't necessarily true, but this analogy isn't as conclusive as you make it sound...

Re: Adversarial image attacks are no joke

#135

Post by GhettoComputers »

tshaddox wrote: post #117
aneutron wrote: post #54

Earlier quoted context omitted.

What you are proposing are what I think would be called a security theater. It gives the illusion of security, but they would absolutely not deter a determined threat actor. The only reason that the water supply isn't poisoned is it's unpractical for a single person to conduct the whole exploit chain: Construct the poison in enough quantities, gain access to facilities supplying the water, and actually throwing the c…

It’s pretty easy for a single person to modify or remove an important street sign. Some kids stole traffic signs and were convicted of manslaughter when someone ran a (missing) stop sign and killed another driver. https://www.washingtonpost.com/archive/politics/1997/06/21/3...

Looks like they were innocent. https://www.law.umich.edu/special/exoneration/Pages/casedeta...

> In 1998, the defense filed a post-conviction motion for a new trial. At a hearing on the motion, several witnesses testified that they had driven by the intersection days before the accident and the stop sign was already down. Some of the witnesses said that after the charges were filed, they reported to both Hillsborough County Sheriff’s detectives and the prosecution that the sign had been down for days, but the information was disregarded. One of the witnesses said she spoke to the prosecutor who disregarded the report and replied that she intended to “burn their ass,” referring to the defendants.

>The motion for a new trial was denied, and the defendants appealed the decision. In March 2001, the Florida Court of Appeals reversed the manslaughter convictions of all three defendants. The court held that the prosecution had made improper comments during closing argument. The court did not reverse the grand theft convictions.

Re: Adversarial image attacks are no joke

#136

Post by littlestymaar »

323 wrote: post #72
marcosdumay wrote: post #31

Earlier quoted context omitted.

I'm pretty sure this would fail to kill people on almost every place you could try it. And if it works somewhere, it's because there are other problems with the road that should be fixed. Human driving is full of redundancies, and there is a clear hierarchy of information. People will not rush into a road full of cars going on the other way, it doesn't matter what the signs say. If your automated driving system doesn…

> People will not rush into a road full of cars going on the other way, it doesn't matter what the signs say. And people would not drive into a river passing through multiple barriers, just because their GPS says so. https://theweek.com/articles/464674/8-drivers-who-blindly-fo... https://indianexpress.com/article/trending/bizarre/driver-in...

A few people taking completely stupid decisions is always going to happen. There's also drunk people or elderly people with poor vision who end up in the wrong direction on the highway. Even if it happens every day on a global scale, it's an incredibly small fraction of drivers. A targeted attack fooling current-technology IA could lead hundreds of cars in the wrong direction at the same time in a single place. It's way worse than what you could imagine tricking humans to do. Putting such brittle systems in charge on the road is irresponsible.

Re: Adversarial image attacks are no joke

#137

Post by sophacles »

inetknght wrote: post #56
marcosdumay wrote: post #31

Earlier quoted context omitted.

I'm pretty sure this would fail to kill people on almost every place you could try it. And if it works somewhere, it's because there are other problems with the road that should be fixed. Human driving is full of redundancies, and there is a clear hierarchy of information. People will not rush into a road full of cars going on the other way, it doesn't matter what the signs say. If your automated driving system doesn…

> People will not rush into a road full of cars going on the other way, it doesn't matter what the signs say. You might want to watch the one-way roads in big cities. It happens a lot more often than you assume. It also is (usually) self-correcting: oncoming traffic will honk, stop, or move around. The offender will (usually) realize their mistake and try to correct. Sometimes, though, that's not enough. Searching "k…

I'd suggest a slight correction here: You don't need the qualifier of 'big cities'. I live in a big town/small city it happens all the time here too. And when I lived in a small town that had a one way drive around the square, it happened a lot there too.

It's so frequent in fact that the barflies at one local place (that has a beer garden from which you can see a one way road) turned it into a drinking game - wrong way car == take a shot.

Re: Adversarial image attacks are no joke

#138

Post by tshaddox »

GhettoComputers wrote: post #135
tshaddox wrote: post #117

Earlier quoted context omitted.

It’s pretty easy for a single person to modify or remove an important street sign. Some kids stole traffic signs and were convicted of manslaughter when someone ran a (missing) stop sign and killed another driver. https://www.washingtonpost.com/archive/politics/1997/06/21/3...

Looks like they were innocent. https://www.law.umich.edu/special/exoneration/Pages/casedeta... > In 1998, the defense filed a post-conviction motion for a new trial. At a hearing on the motion, several witnesses testified that they had driven by the intersection days before the accident and the stop sign was already down. Some of the witnesses said that after the charges were filed, they reported to both Hillsborough…

Fair enough, I'm not very familiar with the details of that case, but my overall points are that 1) it's easy to remove traffic signs (and thoughtless kids probably do it every so often as a dumb joke), 2) this can definitely lead to bad accidents (with human drivers), and 3) the legal system will likely at least consider whether the people who tampered with the traffic signs are culpable for the accidents.

Re: Adversarial image attacks are no joke

#139

Post by littlestymaar »

ummonk wrote: post #123
hermitdev wrote: post #121

Earlier quoted context omitted.

Some are. Some are not. Last week, I was nearly in two accidents on maybe a 1 mile trip to the store from my house. Both times were people pulling out of traffic, ignoring right of way. I prevented the accidents that would have resulted from these two separate idiots. I have also been in over 20 accidents in my 25 years of driving, the vast majority of those having been rear-ended and none were my fault. In my experi…

> I have also been in over 20 accidents in my 25 years of driving, the vast majority of those having been rear-ended and none were my fault. Do you just drive a lot or do you brake too late / too hard? Because an accident rate that high is rather unusual.

Yeah, the “I'm like 5 sigmas deep in the probability distribution but I bear no responsibility” sounds a bit suspicious.

Re: Adversarial image attacks are no joke

#140

Post by mgraczyk »

mannerheim wrote: post #131
mgraczyk wrote: post #7

As somebody who works on computer vision, my general take on these things is that adversarial examples are like poison. It would be fairly easy to add poison to a water supply or the air intake of a large building and kill a large number of people. This rarely happens though. It's ok that water sources, buildings, and people aren't completely immune to poison. The safety requirement isn't that poison can't hurt. Inst…

> 2. People who create and distribute these stickers knowing their purpose should go to prison. This would surely not pass constitutional muster.

You might be surprised to learn that the US government regulates things that are much closer to what you consider "speech". For example,

    > Federal law prohibits the possession with intent to sell or distribute obscenity, to send, ship, or receive obscenity, to import obscenity, and to transport obscenity across state borders for purposes of distribution.
https://www.justice.gov/criminal-ceos/citizens-guide-us-fede...

A specific recent case:

https://www.mtsu.edu/first-amendment/article/167/united-stat...

    > “offers to engage in illegal transactions are categorically excluded from First Amendment protection.”
If it's illegal to posses something, the government can ban offering to sell and distribute it.
Post reply on HN