Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adversarial design printed on a shirt to fool object recognition algorithms

vice.com

Post reply on HN
Print view View original

21–30 of 76 posts

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#21

Post by sumoboy »

scrumper wrote: post #3

Lovely. A bit of social proof hacking could go a long way to making these kind of adversarial designs more common on the streets - hire some actors to go round the city with CV-defeating makeup on, or these T-shirts, or these garments: https://www.vice.com/en_ca/article/qvgpvv/adversarial-fashio... (though I wonder if those designs might be shut down by copyrights on license plate designs?) (As an aside I got a kick…

I'm sure if you avoided trademarked symbols or phrases using just letters in a rectangle it wouldn't be a problem. Time to create a few bumper stickers to fool ALPR's.

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#22

Post by MarkMMullin »

3pt14159 wrote: post #18

Colour me skeptical. There are multiple ways to capture features and the shirt may fool one set of algorithms but I highly doubt they'll fool them all.

This has pretty broad applicability across a wide range of algorithms. The common failure mode when the machine fails to recognize the otherwise normal real face and body indicates that the whole face/skeleton relationship has fallen apart. Defeating this is interesting, as we have enough trouble just trying to recognize faces. To add to this "yes these are faces too but they are not faces too" is probably going to drive some researchers to drink. At the end of the day, this is a common flaw in a lot of deep learning systems, they're very brittle.

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#23

Post by notus »

3pt14159 wrote: post #18

Colour me skeptical. There are multiple ways to capture features and the shirt may fool one set of algorithms but I highly doubt they'll fool them all.

Exactly, this is mostly a gimmick. It works in specific situations, but isn't robust and won't stand the test of time.

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#25

Post by notus »

jadell wrote: post #20
3pt14159 wrote: post #18

Colour me skeptical. There are multiple ways to capture features and the shirt may fool one set of algorithms but I highly doubt they'll fool them all.

Like any good security protocol, this wouldn't be the only line of defense. A combination of adversarial clothing, makeup, hair style, and accessories would be used. And constantly evolving, making countermeasures harder. Security is always reactionary, you can't defend against an attack you've never seen before.

> Security is always reactionary, you can't defend against an attack you've never seen before

Yes you can, that's part of the appeal of applying machine learning to security. They don't rely on things like signatures or existing heuristics to identify things as malicious.

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#26

Post by undersuit »

3pt14159 wrote: post #18

Colour me skeptical. There are multiple ways to capture features and the shirt may fool one set of algorithms but I highly doubt they'll fool them all.

Yeah, it's a static T-shirt, all these systems are just one network update away from a fix. We need dynamic clothing like the SmartShroud from "The Light of Other Days".

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#27

Post by danShumway »

notus wrote: post #25
jadell wrote: post #20

Earlier quoted context omitted.

Like any good security protocol, this wouldn't be the only line of defense. A combination of adversarial clothing, makeup, hair style, and accessories would be used. And constantly evolving, making countermeasures harder. Security is always reactionary, you can't defend against an attack you've never seen before.

> Security is always reactionary, you can't defend against an attack you've never seen before Yes you can, that's part of the appeal of applying machine learning to security. They don't rely on things like signatures or existing heuristics to identify things as malicious.

Machine learning does rely on heuristics, it just builds the heuristics on its own. If it runs into an attack that doesn't use any of the attack vectors it's learned to guard against, it will fail.

Think of it like your body. It learns to identify viruses. Does that mean you're immune from novel viruses or new strains of the flu?

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#28

Post by isthispermanent »

So 100 human sized objects get detected by the algo, and then 1, wearing this t-shirt, that fits most of the parameters doesn't. Very, very easy to adjust the algorithm to account for a t-shirt. This is cute, at best.

It's also then super easy to say that the individual wearing the shirt is likely to try to usurp monitoring. In practice this type of thing will likely make you a more prevalent target for monitoring along the lines of "what do you have to hide?".

Not that I agree at all with large-scale monitoring or think anyone should prove that they don't have something to hide. Only that it paints the target on your back.

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#29

Post by mdorazio »

I’m confused how this helps beyond body recognition. It seems to me that the focus these days is on facial recognition where you would be training your model to look for facial features rather than whatever is on that shirt. Is this supposed to somehow fool that as well by tricking it with false face features or something?

Re: Adversarial design printed on a shirt to fool object recognition algorithms

#30

Post by papln »

This t-shirt defeated 2 CV model, R-CNN and YOLOv2.

We need better deployed testing suites that can test an adversarial model against many popular classifiers, not just 2.

Even so, the paper itsef shows tht their tshirt doesn't make the wearer undetectable, only partially-undetectable. A security system won't ignore you just because it only saw you 10% of the time you were present (unless it's an Uber self-driving car).

Post reply on HN