Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fawkes: Image “Cloaking” for Personal Privacy

sandlab.cs.uchicago.edu

Post reply on HN
Print view View original

61–70 of 122 posts

Re: Fawkes: Image “Cloaking” for Personal Privacy

#61

Post by nxpnsv »

sszz wrote: post #25
Immortal333 wrote: post #12

This is tested on existing models/Face Recognition API which means locked pre-trained models. So, They might have learned way to add pixels such that model outputs very different embedding. This is know issue in deep learning [0][1][2]. I believe, Model trained on cloaked images would defeat its purpose and make this technique useless. [0] Su, Jiawei, Danilo Vasconcellos Vargas, and Kouichi Sakurai. "One pixel attack…

This should be somewhat mitigated if you keep using different cloaking images, no?

There is a problem there that your pics are already out and you can no longer update them....

Re: Fawkes: Image “Cloaking” for Personal Privacy

#62

Post by kccqzy »

katakuchi wrote: post #13

"when someone tries to identify you using an unaltered image of you [...] they will fail." I wonder how this holds up when someone takes a photo of that 'protected image'. I can imagine that if these miniscule pixel-scaled changes aren't visible to the naked eye, my crappy 6 megapixel camera will overlook it as well. If I then proceed to feed that image into my image recognition algorithm, is it still protected?

More importantly, assuming they have a database of such cloaked images, what if someone just applies the same cloaking technique to the image of you? Can they still identify you?

Re: Fawkes: Image “Cloaking” for Personal Privacy

#64

Post by sszz »

nxpnsv wrote: post #61
sszz wrote: post #25

Earlier quoted context omitted.

This should be somewhat mitigated if you keep using different cloaking images, no?

There is a problem there that your pics are already out and you can no longer update them....

If you use a new cloaking image for each picture you upload to social then they will all be embedded in a different location for a given feature extractor and an adversary wouldn’t be able to reverse search for linked pictures—that’s at least my understanding of how the method would need to be used. But if you keep using the same cloaking image, your adversary could definitely learn that process and effectively undo it.

Re: Fawkes: Image “Cloaking” for Personal Privacy

#67

Post by ravenben »

m3kw9 wrote: post #42

What if the AI takes that photo and train with your name labeled on it again?

That is actually what we want to have happen. Fawkes relies on a POISON attack, in that it corrupts the model into learning the wrong thing. So taking our cloaked photos and learning on it is what corrupts the model and provides the protection.

If you're asking: what if the model trains on "uncloaked" images, we talk about that extensively in the paper and provide technical answers and experimental results. Take a look.

Re: Fawkes: Image “Cloaking” for Personal Privacy

#68

Post by lolc »

djyaz1200 wrote: post #19

Looks like it is obfuscating asymmetry. In addition to the intended effect it could also make people more attractive.

Finally a comment that addresses how the cloaking works. All the other comments I've seen here wonder how pixel-level changes can prevent recognition. Well, shifting an eye a few millimeters changes the whole face!

Re: Fawkes: Image “Cloaking” for Personal Privacy

#69

Post by bob1029 »

From a fundamental information theory perspective, if there is still enough information in the image such that we can identify the person, but we cannot identify any delta between 'cloaked' & 'uncloaked' images, then we are just talking about some arbitrary amount of noise that can effectively be ignored.

If a fixed number of bytes can be interpreted visually by a human as a specific human on a reliable basis, there is zero reason a computer cannot accomplish the same. At worst, we are talking about some minor deficit in the current ML libraries that were tested, likely somewhere in pre-filtering of training data. As mentioned in other posts, a low-pass filter is exactly what you would use to side-step this sort of thing.

From a much more sinister perspective, this is potentially even more dangerous than not applying this cloaking process at all. Presumably, there is some way to detect that this process has been applied and that it has certain hallmarks. Assuming it is resilient enough to survive JPEG and other typical compression schemes, I would wonder if perhaps this is a tool to positively identify those who would otherwise want to hide from authorities.

Re: Fawkes: Image “Cloaking” for Personal Privacy

#70

Post by hwillis »

zitterbewegung wrote: post #41
keehun wrote: post #3

I was immediately was curious how it would protect against image compression and/or otherwise de-noising these protection pixel changes. Their paper does address this question, and for those that are interested: Even image compression cannot defeat our cloak. We use progressive JPEG [57], reportedly used by Facebook and Twitter, to compress the images in our dataset. The image quality, as standard by Independent JPEG…

They create a picture that is designed to fool facial recognition systems and then they will test it against JPEG and other image compression techniques and then run the facial recognition on the compressed version and if both of them pass then they don't change the image. If the procedure fails then the GAN will know this and it would change the output so that it would pass both outcomes.

It doesn't use a GAN. I haven't read the paper closely, but is uses a feature extracting network and then tries to modify the features to match a different person. It doesn't try to take advantage of degenerative states in existing networks.

I think it does things like subtly change the shape of your eyes and whatnot. That makes people consistently difficult to pick out of a large set of unmodified faces, but a hu man still finds the images recognizable because they're comparing it to a smaller set.

They didn't create the model by training it on the facial recognizers they tested against, and they tested with several different feature extractors.

Post reply on HN