I've tried this with facenet and it still detects the correct faces I tried without much issue: https://github.com/davidsandberg/facenet/
Fawkes: Image “Cloaking” for Personal Privacy
81–90 of 122 posts
Re: Fawkes: Image “Cloaking” for Personal Privacy
#82I'm not surprised that there are loads of attacks like this. On QI recently (a TV prog in the UK) a series of images were presented showing just how asymmetric our faces are.
Try taking a photo of your face or someone you know with as near symmetric lighting etc as you can manage. Now cut the image vertically and mirror each half and compare visually.
Frightening isn't it?
Now add fancy patterns that can be hidden within an image that eyes miss but algos don't. AI does not see the way you and I do. It can't. AI can be twiddled constantly to get it into line with what we perceive and we could call that evolution. In 200M years it might be quite good.
I suspect that progress will be faster than that but those machines can't type on a keyboard balanced on its knee whilst drinking wine and admiring a landscape with a setting sun whilst worrying about how to shop tomorrow, now masks are compulsory. What's the SO up to? The TV is showing crap and a new Netflix series is available but I can't be arsed ...
The march of our robot overlords is unlikely soon.
Re: Fawkes: Image “Cloaking” for Personal Privacy
#83...and someone will run their training data through this and retrain a model that defeats it.
Re: Fawkes: Image “Cloaking” for Personal Privacy
#84> You can then use these "cloaked" photos as you normally would, sharing them on social media, sending them to friends, printing them or displaying them on digital devices, the same way you would any other photo. The difference, however, is that if and when someone tries to use these photos to build a facial recognition model, "cloaked" images will teach the model an highly distorted version of what makes you look like you. The cloak effect is not easily detectable, and will not cause errors in model training. However, when someone tries to identify you using an unaltered image of you (e.g. a photo taken in public), and tries to identify you, they will fail.
So, if I adopt this and upload only cloaked images on social media, and the people I normally interact with also do the same, then facial recognition will be able to detect me based on someone showing the system that I’m present in the photo (even though it identifies me as the distorted version)?
If the above understanding is true, then even law enforcement could cloak all the photos they have and try to match captures with their raw photo set and the cloaked photo set to narrow it down for a human?
What am I missing?
Re: Fawkes: Image “Cloaking” for Personal Privacy
#85I'm not surprised that there are loads of attacks like this. On QI recently (a TV prog in the UK) a series of images were presented showing just how asymmetric our faces are. Try taking a photo of your face or someone you know with as near symmetric lighting etc as you can manage. Now cut the image vertically and mirror each half and compare visually. Frightening isn't it? Now add fancy patterns that can be hidden wi…
Why cut and mirror? Can't you just mirror? Wouldn't this be the same as just looking at someone's face in the mirror?
Re: Fawkes: Image “Cloaking” for Personal Privacy
#86> You can then use these "cloaked" photos as you normally would, sharing them on social media, sending them to friends, printing them or displaying them on digital devices, the same way you would any other photo. The difference, however, is that if and when someone tries to use these photos to build a facial recognition model, "cloaked" images will teach the model an highly distorted version of what makes you look li…
Perhaps it isn't predictable? I would hope that the face recognizer can't just cloak the uncloaked images and then get the match.
Re: Fawkes: Image “Cloaking” for Personal Privacy
#87> You can then use these "cloaked" photos as you normally would, sharing them on social media, sending them to friends, printing them or displaying them on digital devices, the same way you would any other photo. The difference, however, is that if and when someone tries to use these photos to build a facial recognition model, "cloaked" images will teach the model an highly distorted version of what makes you look li…
My guess is that the changes it makes - presumably moving the standard face landmarks - are different for each picture you run through it, so multiple images of the same face will not be recognised as the same face.
(But I'm not sure, and have downloaded the paper and the apps to read and experiment with...)
Re: Fawkes: Image “Cloaking” for Personal Privacy
#88Earlier quoted context omitted.
This should be somewhat mitigated if you keep using different cloaking images, no?
But the model will eventually be updated to detect and process the new cloaking images. So, to stay ahead, you decide to create a model that automatically generates different cloaking images, and... The whole system is now just a GAN : https://en.wikipedia.org/wiki/Generative_adversarial_network
I think there's a (hopefully strongly privacy preserving) combinatorial explosion here though. If current models can be trained to accurately-enough recognise me with, say, 100 training images - this tool might produce unique enough perturbations to require 100 images for each of the possible perturbations, potentially requiring you to train your new model using tens of thousands or millions of cloaked versions of the 100 images for each of the targets in your training set.
(If I were these researchers I'd totally be reaching out to AWS/Azure/GCE for additional research funding... )
Re: Fawkes: Image “Cloaking” for Personal Privacy
#89Earlier quoted context omitted.
This should be somewhat mitigated if you keep using different cloaking images, no?
There is a problem there that your pics are already out and you can no longer update them....
The FAQ there addresses that, suggesting you can "dilute down" the ratio of normal-to-cloaked images in the public data sets the model creators train on, and hence reduce their future accuracy.
(So now you just need to somehow get as many cloaked photos of yourself uploaded and tagged to FB as they've collected in the last decade or so...)
Re: Fawkes: Image “Cloaking” for Personal Privacy
#90Once this technique gets enough attention, a detector for it will be built. Even if the face cannot be recognized, a profile with such picture may be flagged for more scrutiny. This reminds me of using TOR that hides what you visit, and yet likely puts you into a watchlist for surveillance. I think that a simpler and more robust strategy to achieve good privacy is avoid posting personal information online and social…
"Hey Bob? Check out this lykahb person. There's something _off_ about them. No Facebook, no Twitter, not even LinkedIn. Probably up to something, we should keep an eye on them. Add them to the list." -- some NSA/GRU/MSS/Mossad contractor