Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fawkes: Image “Cloaking” for Personal Privacy

sandlab.cs.uchicago.edu

Post reply on HN
Print view View original

41–50 of 122 posts

Re: Fawkes: Image “Cloaking” for Personal Privacy

#41

Post by zitterbewegung »

keehun wrote: post #3

I was immediately was curious how it would protect against image compression and/or otherwise de-noising these protection pixel changes. Their paper does address this question, and for those that are interested: Even image compression cannot defeat our cloak. We use progressive JPEG [57], reportedly used by Facebook and Twitter, to compress the images in our dataset. The image quality, as standard by Independent JPEG…

They create a picture that is designed to fool facial recognition systems and then they will test it against JPEG and other image compression techniques and then run the facial recognition on the compressed version and if both of them pass then they don't change the image.

If the procedure fails then the GAN will know this and it would change the output so that it would pass both outcomes.

Re: Fawkes: Image “Cloaking” for Personal Privacy

#43

Post by baxtr »

Tarq0n wrote: post #5

While this seems to work against several current techniques, there's no guarantee it will work against all of them. It also offers no guarantees against future developments, and anything you put on the public internet is likely to be retained forever. Because of this I'd consider it an interesting proof of concept, but not something anyone should use as a privacy tool. You could consider it in cases where you're forc…

Yeah sure, I agree. At the same time: isn’t that exactly the kind of perpetual armament that’s going on in many fields?

Re: Fawkes: Image “Cloaking” for Personal Privacy

#44

Post by scribu »

yalogin wrote: post #24
scribu wrote: post #22

Earlier quoted context omitted.

In theory, yes, you can feed these new distorted images into a new model and defeat the cloaking. In practice, building new training sets is much more expensive than slightly changing the cloaking algorithm.

That is awesome. I need to read the paper. Really promising. Are you the author btw?

Not the author; just a fellow ML practitioner.

Re: Fawkes: Image “Cloaking” for Personal Privacy

#45

Post by skylanh »

akersten wrote: post #8

Color me extremely skeptical. A low-pass filter will make short work of any "tiny, pixel-level" changes designed to thwart ML. After all, one of the most tell-tale identifiers (space between eyes/nose/mouth) is still plainly observable and unaltered in the "cloaked" image. If a human's neural network can correctly correlate the before/after examples, so can a computer's. They might have found an issue with some moder…

This was my first impression as well, except with less knowledge on the subject.

Someone, somewhere said "huh...", and placed another filter into the pipeline to handle these types of images.

While looking for something to sound smart wrt the tank-training myth, I found this interesting page: https://www.gwern.net/Tanks "The Neural Net Tank Urban Legend"

And interestingly, looking at the link for "superresolution needing learned downscalers" found this: https://arxiv.org/abs/1907.12904 "Learned Image Downscaling for Upscaling using Content Adaptive Resampler", code available at https://github.com/sunwj/CAR

So, IDK, seems like this Fawkes approach will be an interesting paper.

Re: Fawkes: Image “Cloaking” for Personal Privacy

#46

Post by DabbyDabberson »

akersten wrote: post #8

Color me extremely skeptical. A low-pass filter will make short work of any "tiny, pixel-level" changes designed to thwart ML. After all, one of the most tell-tale identifiers (space between eyes/nose/mouth) is still plainly observable and unaltered in the "cloaked" image. If a human's neural network can correctly correlate the before/after examples, so can a computer's. They might have found an issue with some moder…

> If a human's neural network can correctly correlate the before/after examples, so can a computer's.

color _me_ skeptical, but this is like saying we have functioning AGI; that artificial NNs are the same as the ones we have in our skulls. This to me, is an effect of the over-anthropomorphization of machine learning. It's a bad intuition to have.

However, I do agree. This is just one step in an arms race, and one iteration from being worthless.

Re: Fawkes: Image “Cloaking” for Personal Privacy

#47

Post by antpls »

sszz wrote: post #25
Immortal333 wrote: post #12

This is tested on existing models/Face Recognition API which means locked pre-trained models. So, They might have learned way to add pixels such that model outputs very different embedding. This is know issue in deep learning [0][1][2]. I believe, Model trained on cloaked images would defeat its purpose and make this technique useless. [0] Su, Jiawei, Danilo Vasconcellos Vargas, and Kouichi Sakurai. "One pixel attack…

This should be somewhat mitigated if you keep using different cloaking images, no?

But the model will eventually be updated to detect and process the new cloaking images. So, to stay ahead, you decide to create a model that automatically generates different cloaking images, and... The whole system is now just a GAN : https://en.wikipedia.org/wiki/Generative_adversarial_network
Post reply on HN