Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fawkes: Image “Cloaking” for Personal Privacy

sandlab.cs.uchicago.edu

Post reply on HN
Print view View original

91–100 of 122 posts

Re: Fawkes: Image “Cloaking” for Personal Privacy

#91

Post by akersten »

woah wrote: post #78
akersten wrote: post #8

Color me extremely skeptical. A low-pass filter will make short work of any "tiny, pixel-level" changes designed to thwart ML. After all, one of the most tell-tale identifiers (space between eyes/nose/mouth) is still plainly observable and unaltered in the "cloaked" image. If a human's neural network can correctly correlate the before/after examples, so can a computer's. They might have found an issue with some moder…

From the linked article: Q: Can't you just apply some filter, or compression, or blurring algorithm, or add some noise to the image to destroy image cloaks? A: As counterintuitive as this may be, the high level answer is no simple tools work to destroy the perturbation that form image cloaks. To make sense of this, it helps to first understand that Fawkes does not use high-intensity pixels, or rely on bright patterns…

It was pretty frustrating that they did not readily offer any example images for inspection, so against my better judgement I downloaded their binaries to run some experiements.

First, a source image at an approximate resolution that you might find on a social networking site: https://imgur.com/a/9szcC1m

Text output of the tool, which ran for about 3 minutes: https://imgur.com/a/fZtfrmm

The resulting cloaked image: https://imgur.com/a/OSHXdbO

I applied a difference filter between the two images in Photoshop, to show an example of the actual pertubations performed: https://imgur.com/a/q4zC7Ms

Since it's hard to see, I compressed the output to highlight what the program actually changed. It does seem like there is a good amount of disturbance to the image: https://imgur.com/a/1Sx68o3

Now, the real test. First, a Google reverse image search for the original file - identification is pretty bang-on: https://imgur.com/a/5HJwjPx

A Google reverse image search for the cloaked file: https://imgur.com/a/QByXBfS

The only difference I'm seeing is a few images that are one or two images swapped in the "visually similar images" category.

So, I figured that that's the "best case" for the cloaked image - giving the search algorithm the full, unfiltered data, and the program still failed to disguise it. For fun, I thought I would use a "low-pass filter" (Google Lens pointed at my computer screen) as well, just for thoroughness. And the result surprised me!

Here's Google Lens pointed at my screen with the original image open: https://imgur.com/a/1BVRFG0

And here's Google Lens pointed at my screen with the cloaked image open: https://imgur.com/a/uoppuit

So, it would seem that the algorithm's distortions more effectively come through in worse quality images! But, based on my full-resolution result, I wouldn't trust it to disguise something that is being directly uploaded to a social network.

Now, one important note is that reverse image search is probably not using a facial recognition model, but more like image chunk hashing - although I would also consider that something a privacy tool should defend against, which is why I included it.

All in all, very interesting and thanks for convincing me that I should actually test it out.

Re: Fawkes: Image “Cloaking” for Personal Privacy

#92

Post by angry_octet »

cetra3 wrote: post #81

I've tried this with facenet and it still detects the correct faces I tried without much issue: https://github.com/davidsandberg/facenet/

How many faces were in the database you tested with? Face re-ID is much simpler for a small cohort.

Re: Fawkes: Image “Cloaking” for Personal Privacy

#93

Post by Bjartr »

akersten wrote: post #91
woah wrote: post #78

Earlier quoted context omitted.

From the linked article: Q: Can't you just apply some filter, or compression, or blurring algorithm, or add some noise to the image to destroy image cloaks? A: As counterintuitive as this may be, the high level answer is no simple tools work to destroy the perturbation that form image cloaks. To make sense of this, it helps to first understand that Fawkes does not use high-intensity pixels, or rely on bright patterns…

It was pretty frustrating that they did not readily offer any example images for inspection, so against my better judgement I downloaded their binaries to run some experiements. First, a source image at an approximate resolution that you might find on a social networking site: https://imgur.com/a/9szcC1m Text output of the tool, which ran for about 3 minutes: https://imgur.com/a/fZtfrmm The resulting cloaked image: h…

There's also a chance Google image search is looking at the filename of the image to get a bit more context. Does the reverse image search of the cloaked image still work if you rename it something unrelated?

Re: Fawkes: Image “Cloaking” for Personal Privacy

#94

Post by bigiain »

kccqzy wrote: post #62
katakuchi wrote: post #13

"when someone tries to identify you using an unaltered image of you [...] they will fail." I wonder how this holds up when someone takes a photo of that 'protected image'. I can imagine that if these miniscule pixel-scaled changes aren't visible to the naked eye, my crappy 6 megapixel camera will overlook it as well. If I then proceed to feed that image into my image recognition algorithm, is it still protected?

More importantly, assuming they have a database of such cloaked images, what if someone just applies the same cloaking technique to the image of you? Can they still identify you?

That's making a pretty lazy assumption that even a quick read of the original article leads me to be sure it's incorrect.

There's quite a lot of comments here that stink of Dunning Kruger candidates, who read the headline and first paragraph, then just started typing their random "wisdom" assuming they're smarter and better informed that the team of PHD researchers who wrote the paper being discussed. (Am I just overly grumpy and judgemental today? Was HN always this bad?)

Re: Fawkes: Image “Cloaking” for Personal Privacy

#96

Post by aussieguy1234 »

Tarq0n wrote: post #5

While this seems to work against several current techniques, there's no guarantee it will work against all of them. It also offers no guarantees against future developments, and anything you put on the public internet is likely to be retained forever. Because of this I'd consider it an interesting proof of concept, but not something anyone should use as a privacy tool. You could consider it in cases where you're forc…

its an arms race, like alot of things

Re: Fawkes: Image “Cloaking” for Personal Privacy

#97

Post by akersten »

Bjartr wrote: post #93
akersten wrote: post #91

Earlier quoted context omitted.

It was pretty frustrating that they did not readily offer any example images for inspection, so against my better judgement I downloaded their binaries to run some experiements. First, a source image at an approximate resolution that you might find on a social networking site: https://imgur.com/a/9szcC1m Text output of the tool, which ran for about 3 minutes: https://imgur.com/a/fZtfrmm The resulting cloaked image: h…

There's also a chance Google image search is looking at the filename of the image to get a bit more context. Does the reverse image search of the cloaked image still work if you rename it something unrelated?

Fair shake, this time I cropped the image too so that it wasn't looking at any of the boundary to identify it either.

Here's the original image: https://imgur.com/a/Td4rhoy

And the cloaked: https://imgur.com/a/cPCiCZo

These were both saved as JPG with compression level 8/12. I searched for the cloaked crop (96.jpg) first this time: https://imgur.com/a/FSehQWO

And the original crop (10.jpg) next: https://imgur.com/a/yx4jF0B

This time, Google reverse image search did better at identifying the name of the singer in the cloaked image, instead of just giving the band name for the uncloaked.

Not super scientific since we don't really know what's going on behind-the-scenes with Google reverse image search, but it's certainly one adversary that doesn't seem to be easily fooled if there are other images of "you" out there for it to find. I also tried these small crops in Google Lens with less success (I got unrelated portraits for both images, cloaked or not).

Re: Fawkes: Image “Cloaking” for Personal Privacy

#98

Post by angry_octet »

I just tried it on myself and it produces some wierd colour distortions, particularly around the eyebrows, even with the default 'low' setting (50 iterations, threshold 0.003), so I'm not sure people will be happy using it. Kind of looks like I've been attacked by a bad eyebrow pencil. Also looks a bit like a picture printed on thin magazine paper (something trashy like OK! Magazine) held up to the light, so the image from the other side bleeds through.

If you tweak the values a bit lower it doesn't look so bad, but of course I haven't tested it with an array of DL algorithms.

Re: Fawkes: Image “Cloaking” for Personal Privacy

#99

Post by kevingadd »

akersten wrote: post #91
woah wrote: post #78

Earlier quoted context omitted.

From the linked article: Q: Can't you just apply some filter, or compression, or blurring algorithm, or add some noise to the image to destroy image cloaks? A: As counterintuitive as this may be, the high level answer is no simple tools work to destroy the perturbation that form image cloaks. To make sense of this, it helps to first understand that Fawkes does not use high-intensity pixels, or rely on bright patterns…

It was pretty frustrating that they did not readily offer any example images for inspection, so against my better judgement I downloaded their binaries to run some experiements. First, a source image at an approximate resolution that you might find on a social networking site: https://imgur.com/a/9szcC1m Text output of the tool, which ran for about 3 minutes: https://imgur.com/a/fZtfrmm The resulting cloaked image: h…

GIS is specifically designed to be good at finding similar images so it's going to work great for your test case. Facial recognition algorithms are solving a different problem.

If they were promising that cloaking would work well on GIS, that'd be a different matter. I can imagine wanting your images to not show up on GIS (because people would use them to try and find the source image on your profile, or something) but it's a different set of constraints at that point.

For cloaking a big use case would be "I took a selfie with a friend and want to share it on my instagram" and your goal is for that instagram selfie to not automatically connect with, for example, a surveillance photo of you at a protest. GIS is obviously not relevant to that scenario.

Re: Fawkes: Image “Cloaking” for Personal Privacy

#100

Post by hadeson »

akersten wrote: post #97
Bjartr wrote: post #93

Earlier quoted context omitted.

There's also a chance Google image search is looking at the filename of the image to get a bit more context. Does the reverse image search of the cloaked image still work if you rename it something unrelated?

Fair shake, this time I cropped the image too so that it wasn't looking at any of the boundary to identify it either. Here's the original image: https://imgur.com/a/Td4rhoy And the cloaked: https://imgur.com/a/cPCiCZo These were both saved as JPG with compression level 8/12. I searched for the cloaked crop (96.jpg) first this time: https://imgur.com/a/FSehQWO And the original crop (10.jpg) next: https://imgur.com/a/y…

Yandex seem to give even more precise matches than Google. Maybe image of someone less popular would work better.
Post reply on HN