A wearable captcha!
Adversarial design printed on a shirt to fool object recognition algorithms
31–40 of 76 posts
Re: Adversarial design printed on a shirt to fool object recognition algorithms
#32Where can I buy this merch?
Re: Adversarial design printed on a shirt to fool object recognition algorithms
#33The arXiv paper contains images of the shirts and methodologies:
Re: Adversarial design printed on a shirt to fool object recognition algorithms
#34So 100 human sized objects get detected by the algo, and then 1, wearing this t-shirt, that fits most of the parameters doesn't. Very, very easy to adjust the algorithm to account for a t-shirt. This is cute, at best. It's also then super easy to say that the individual wearing the shirt is likely to try to usurp monitoring. In practice this type of thing will likely make you a more prevalent target for monitoring al…
Ya, I immediately thought of the IlIlIl licence plate xkcd. https://xkcd.com/1105/
Re: Adversarial design printed on a shirt to fool object recognition algorithms
#35Straight out of William Gibson's "Zero History"!
In Zero History, the purpose of the shirt was not to fool the algorithm, but to trip a deep "gentlemen's agreement" between intelligence agencies to make invisible anyone bearing a certain pattern in order to protect the intelligence apparatus.
It will always be difficult to sustainably defeat recognition algorithms and I expect this to be an arms race along the same lines as other counter-surveillance techniques.
Gibson's suggestion that deeply coded and secret exceptions to mass surveillance might be used to protect state actors seems to me a plausible and concerning aspect of these developments.
Re: Adversarial design printed on a shirt to fool object recognition algorithms
#36This reminds me of that recent article about how zebra stripes have been shown to reduce bug bites when painted on cows: https://news.ycombinator.com/item?id=21201807. Probably a similar effect on object detection algorithms.
Re: Adversarial design printed on a shirt to fool object recognition algorithms
#37One of the most common uses of this tech in the US is automatic license plate readers.
Without getting into a debate about expectations of privacy on public roads vs. building a perpetual government database that tracks where every car is effectively at all times of day, another application of this tech would be a bumper decal.
I think most reasonable people would agree obscuring the license plate on a public road is not the solution (well, with the exception of Florida Man who racked up a $1MM fine when he was finally caught doing that through toll booths for a year), but a decal like this wouldn't interfere with any officer's human duties.
Re: Adversarial design printed on a shirt to fool object recognition algorithms
#38So 100 human sized objects get detected by the algo, and then 1, wearing this t-shirt, that fits most of the parameters doesn't. Very, very easy to adjust the algorithm to account for a t-shirt. This is cute, at best. It's also then super easy to say that the individual wearing the shirt is likely to try to usurp monitoring. In practice this type of thing will likely make you a more prevalent target for monitoring al…
> Very, very easy to adjust the algorithm for a t-shirt.
The operative point here is not 'a shirt', but a visual pattern that tricks deep learning-style classifiers into wildly misidentifying something. There's no 'very easy' way to counteract that other than retraining on a new dataset or switching entirely away from a deep learning system.
Re: Adversarial design printed on a shirt to fool object recognition algorithms
#39It's going to be fun to see this whole recognition proof clothing turn into a low-key "war" as states demand better recognition systems that can bypass this kind of thing, and privacy activists keep developing new ways of fooling AI.
Re: Adversarial design printed on a shirt to fool object recognition algorithms
#40Earlier quoted context omitted.
Like any good security protocol, this wouldn't be the only line of defense. A combination of adversarial clothing, makeup, hair style, and accessories would be used. And constantly evolving, making countermeasures harder. Security is always reactionary, you can't defend against an attack you've never seen before.
> Security is always reactionary, you can't defend against an attack you've never seen before Yes you can, that's part of the appeal of applying machine learning to security. They don't rely on things like signatures or existing heuristics to identify things as malicious.
If you train your security on parallel lines, and I come in with circles, I've just defeated your security. Machine learning only learns how to categorize things into predetermined categories. If I come in with a novel category it's never seen before, the best it can do is guess, and most likely, it will be worse at guessing than random chance.