Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adversarial design printed on a shirt to fool object recognition algorithms

vice.com

Post reply on HN
Print view View original

41–50 of 76 posts

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#41

Post by notus »

jadell wrote: post #40
notus wrote: post #25

Earlier quoted context omitted.

> Security is always reactionary, you can't defend against an attack you've never seen before Yes you can, that's part of the appeal of applying machine learning to security. They don't rely on things like signatures or existing heuristics to identify things as malicious.

If you train your security on parallel lines, and I come in with circles, I've just defeated your security. Machine learning only learns how to categorize things into predetermined categories. If I come in with a novel category it's never seen before, the best it can do is guess, and most likely, it will be worse at guessing than random chance.

Except nobody would train just on parallel lines. They use a wide array of different types of data so the model can make generalizations about things it hasn't seen before.

> Machine learning only learns how to categorize things into predetermined categories.

This is just one type of machine learning called classification, there are others like regression and clustering which can be combined to create more robust models. Look at the technology behind Cylance's product which identifies files as malicious or not pre-execution. They are not just using classification.

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#42

Post by notus »

danShumway wrote: post #27
notus wrote: post #25

Earlier quoted context omitted.

> Security is always reactionary, you can't defend against an attack you've never seen before Yes you can, that's part of the appeal of applying machine learning to security. They don't rely on things like signatures or existing heuristics to identify things as malicious.

Machine learning does rely on heuristics, it just builds the heuristics on its own. If it runs into an attack that doesn't use any of the attack vectors it's learned to guard against, it will fail. Think of it like your body. It learns to identify viruses. Does that mean you're immune from novel viruses or new strains of the flu?

I think it was implied that I meant heuristics that humans have added themselves. The point of it all is to allow models to make generalizations about things it hasn't seen before. This can be done with a combination of supervised and unsupervised techniques.

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#47

Post by nathancahill »

nathancahill wrote: post #46
jmartinpetersen wrote: post #45

You can fool all the AIs some of the time, and some of the AIs all the time, but you cannot fool all the AIs all the time.

Carl Sandburg said that.

Deep dive in to the backstory behind the quote:

http://www.taxhelp.com/lincoln.html

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#48

Post by dTal »

crooked-v wrote: post #38
isthispermanent wrote: post #28

So 100 human sized objects get detected by the algo, and then 1, wearing this t-shirt, that fits most of the parameters doesn't. Very, very easy to adjust the algorithm to account for a t-shirt. This is cute, at best. It's also then super easy to say that the individual wearing the shirt is likely to try to usurp monitoring. In practice this type of thing will likely make you a more prevalent target for monitoring al…

> Very, very easy to adjust the algorithm for a t-shirt. The operative point here is not 'a shirt', but a visual pattern that tricks deep learning-style classifiers into wildly misidentifying something. There's no 'very easy' way to counteract that other than retraining on a new dataset or switching entirely away from a deep learning system.

Surely all you have to do is train a "trippy shirt" detector, and run it in parallel?
Post reply on HN