Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adversarial design printed on a shirt to fool object recognition algorithms

vice.com

Post reply on HN
Print view View original

51–60 of 76 posts

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#51

Post by danShumway »

notus wrote: post #42
danShumway wrote: post #27

Earlier quoted context omitted.

Machine learning does rely on heuristics, it just builds the heuristics on its own. If it runs into an attack that doesn't use any of the attack vectors it's learned to guard against, it will fail. Think of it like your body. It learns to identify viruses. Does that mean you're immune from novel viruses or new strains of the flu?

I think it was implied that I meant heuristics that humans have added themselves. The point of it all is to allow models to make generalizations about things it hasn't seen before. This can be done with a combination of supervised and unsupervised techniques.

> heuristics that humans have added themselves

I don't think this is a meaningful distinction. Who cares whether the new heuristic is being added by a machine or a human?

You still need to keep feeding the neural network data to learn from, and it will still choke when it sees novel data that doesn't align with the heuristics it developed.

That's the entire reason adversarial AI works. The reason the Trippy T-shirt makes you invisible to some current AI systems is because it exploits the heuristics they've built using data that these systems are unfamiliar with and haven't learned to process yet. If it was possible to build an AI system that could defend against novel attacks, the Trippy T-Shirt wouldn't be able to fool them.

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#52

Post by aasasd »

scrumper wrote: post #3

Lovely. A bit of social proof hacking could go a long way to making these kind of adversarial designs more common on the streets - hire some actors to go round the city with CV-defeating makeup on, or these T-shirts, or these garments: https://www.vice.com/en_ca/article/qvgpvv/adversarial-fashio... (though I wonder if those designs might be shut down by copyrights on license plate designs?) (As an aside I got a kick…

> shut down by copyrights on license plate designs

Afaik anything made by the government is expressly public domain in the US, but dunno about state govs. Unless, anyway, the state buys a design from a private company, at which point my law knowledge ends.

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#53

Post by aasasd »

3pt14159 wrote: post #18

Colour me skeptical. There are multiple ways to capture features and the shirt may fool one set of algorithms but I highly doubt they'll fool them all.

Yup, I have the same problem with DPI circumvention software. We know that it fools more-or-less widely available open and commercial DPI suites. Does it fool systems that are made specifically for deployment on ISPs as mandated by the government? Who the hell knows, aside from those working with the systems. The system may even not shut down your requests, but mark you as a weirdo to keep an eye on.

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#54

Post by Kalium »

crooked-v wrote: post #38
isthispermanent wrote: post #28

So 100 human sized objects get detected by the algo, and then 1, wearing this t-shirt, that fits most of the parameters doesn't. Very, very easy to adjust the algorithm to account for a t-shirt. This is cute, at best. It's also then super easy to say that the individual wearing the shirt is likely to try to usurp monitoring. In practice this type of thing will likely make you a more prevalent target for monitoring al…

> Very, very easy to adjust the algorithm for a t-shirt. The operative point here is not 'a shirt', but a visual pattern that tricks deep learning-style classifiers into wildly misidentifying something. There's no 'very easy' way to counteract that other than retraining on a new dataset or switching entirely away from a deep learning system.

As I understand it, adversarial designs generally work on one specific recognition system. So working around this attack would be very achievable with three or more recognition systems and a consensus check.

This particular paper is based around attacking YOLOv2.

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#55

Post by polishTar »

Kalium wrote: post #54
crooked-v wrote: post #38

Earlier quoted context omitted.

> Very, very easy to adjust the algorithm for a t-shirt. The operative point here is not 'a shirt', but a visual pattern that tricks deep learning-style classifiers into wildly misidentifying something. There's no 'very easy' way to counteract that other than retraining on a new dataset or switching entirely away from a deep learning system.

As I understand it, adversarial designs generally work on one specific recognition system. So working around this attack would be very achievable with three or more recognition systems and a consensus check. This particular paper is based around attacking YOLOv2.

I think these types of adversarial attacks are even easier to foil than that because they're specific to one particular set of weights. Even really really small changes in the training data or model could invalidate the attack if I understand correctly.

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#57

Post by Jaygles »

mdorazio wrote: post #29

I’m confused how this helps beyond body recognition. It seems to me that the focus these days is on facial recognition where you would be training your model to look for facial features rather than whatever is on that shirt. Is this supposed to somehow fool that as well by tricking it with false face features or something?

I can imagine a scenario where a system doesn't attempt to look at a face before it determines there's a full human in the frame.

Ultimately if a system is designed to only look at faces then this method would likely not be effective.

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#59

Post by alep »

We also tested fooling YoloV2 using t-shirts, but as mentioned in the paper, we got mixed results. You can fool the object detection only if you get a frontal exposure to the camera without any torsion / rotation / bending of the t-shirt, which is pretty hard in real life. Would be interesting to see if you can train adversarial examples robust to multiple angles. We thought to put these t-shirts out for sale for fun and to send a message: #donottrack. https://stealth.cool

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#60

Post by cc439 »

The joke about Juggalo facepaint is both true and funny but I think there is some actual merit to that idea. Camo clothing (and I don't mean the kind you see everyone wearing at rural WalMartss) goes in and out of fashion every couple years. Military-style jackets, boots, and caps (think of a stereotypical anarchist style) are also perennially in style with certain crowds. I don't think it's too far fetched to imagine a future where camo facepaint becomes fashionable enough to be widespread, there's also a lot of artistic potential available in non-traditional patterns and colors.

I can't really see a way for AI cameras to get around properly applied facepaint, especially varieties that are IR absorbent or reflective. I hold the human brain in very high regard when it comes to pattern/symbol/shape recognition and if facepainting techniques are good enough to trick human visual processing, it's going to be good enough to fool any existing AI. For an example of what I mean by proper technique, refer to this video: https://youtu.be/YpzUr3twW4Q

The trick is in getting enough people to adopt such a strategy that you can't be identified through simple exclusion. I think the idea of camo/other facepaint isn't so foreign and unappealing as to never come into common fashion.

Post reply on HN