Hear me out: blockchain cameras? \s But maybe we'll just have to limit cameras outputting NFT videos, or signed frames as the only source of trusted media
Working on it! Main comment here:
A new way to detect ‘deepfake’ picture editing
81–90 of 92 posts
Re: A new way to detect ‘deepfake’ picture editing
#82While all this stuff around deep fakes and its detection is interesting I'm still left wondering why we don't use cryptography to prove if an image is authentic or not.
I had the same realization several months ago. In the arms race between deepfake detection and generation it's widely understood that generation will win. That's why I started working on Tovera[0].
Main comment here:
Re: A new way to detect ‘deepfake’ picture editing
#83It's hard to see this as much anything but snake oil. While the technique allegedly allows you to target multiple models, it still requires you to know what adversarial model you're targeting before the fact. If the adversary sees there's visible artifacts they'll just change a few parameters or use a different model and be good to go none the less. In short, while this might be academically interesting, I can't help…
A few weeks ago, there was a post here about that guy who deepfaked female profile pictures. He showed that the visible artifacts go away by merely scaling up the number of parameters in the model. Give it a few years and these fake images will be impossible to detect unless we agree on some way of cryptographically signing images from real cameras.
My new startup Tovera[0] is working on a verification approach that (with some tweaking and user feedback) should be usable by anyone without some of the pesky and potentially privacy concerning aspects of a key/cert based approach. More in my comment here:
Re: A new way to detect ‘deepfake’ picture editing
#84Seems like compression is not such a bad deterrent for adversarial deepfakes.
Re: A new way to detect ‘deepfake’ picture editing
#85Earlier quoted context omitted.
>Ssl certificates have been facing this problem for years. That's an entirely different, and much easier problem. In the case of SSL you are not worried about an attacker obtaining a certificate for any host, just the ones you care about. E.g. an attacker getting a certificate for facebook.com would be catastrophic, but an attacker getting a certificate for a website that no one uses would be a non-issue. For the cas…
Every camera will have a unique key so a single key leaked is still no issue. It's easy to maintain a blacklist - similarly how CRLs work.
So now we are essentially bricking the devices of real users? Without doing anything meaningful to real attackers who just grab the next key out of their list of 10 million stolen from insecure androids.
Re: A new way to detect ‘deepfake’ picture editing
#86Earlier quoted context omitted.
>Ssl certificates have been facing this problem for years. That's an entirely different, and much easier problem. In the case of SSL you are not worried about an attacker obtaining a certificate for any host, just the ones you care about. E.g. an attacker getting a certificate for facebook.com would be catastrophic, but an attacker getting a certificate for a website that no one uses would be a non-issue. For the cas…
Surely the threat model remains the same. being able to forge photos from a camera held by the LAPD forensics lab, or Reuters would be more "damaging" than having my camera hardware key. I think there is scope for a simple self regulation here to start us off. I would love to see Reuters or the BBC start to publish their raw footage with the hashes. It is a question of starting the ball rolling
Not really because this is being proposed as a solution to solve people using fake photos on dating apps which means it has to work on every single consumer device to actually work.
Re: A new way to detect ‘deepfake’ picture editing
#87While all this stuff around deep fakes and its detection is interesting I'm still left wondering why we don't use cryptography to prove if an image is authentic or not.
I had the same realization several months ago. In the arms race between deepfake detection and generation it's widely understood that generation will win. That's why I started working on Tovera[0]. Main comment here: https://news.ycombinator.com/item?id=28401256#28407184 [0] https://tovera.com
Interesting approach! I will read into it.
Re: A new way to detect ‘deepfake’ picture editing
#88While all this stuff around deep fakes and its detection is interesting I'm still left wondering why we don't use cryptography to prove if an image is authentic or not.
Will signing come from the camera? At that point you run into DRM. Besides, what will the camera sign? The produced JPG? The raw file? What about rescaling? Do we want ZKPs that a JPG was achieved by nothing more than re-scaling and tone-mapping another JPG or RAW file? Those ZKPs are going to be massively big and slow to verify.
My main concern is the faking of official statements by public figures where having those signed addresses the issue of deep fakes. Of course there has to be infrastructure in place for this and people must know that you can verify them.
If we get to a point in society where unsigned images, or images signed from a questionable source are looked at skeptically, then that alone is a step in the right direction. I don't know the final solution and I don't claim to know it, but cryptography seems like a step in the right direction.
Re: A new way to detect ‘deepfake’ picture editing
#89Earlier quoted context omitted.
I guess there'd have to be certified editors, which would be trusted to allow only "safe" operations on images (crop, white balance, levels, etc) and would sign the resulting jpegs themselves alongside the signature from the camera. It doesn't really seem likely at all to work in practice.
Editors include deep fake technology now. Removing people from a scene has been a common technique for a long time and generative models just have made this easier (e.g. usage: you want the background/scenery but it is a well traveled tourist destination so you can't get a photo without people in it.[0]) [0] Normally you accomplish this by taking several photos and combining them
Sure, you'd have to have a "safe mode" or something for Photoshop that would lock you out of almost everything that could be used to alter the semantic meaning of an image. Edits made in that mode would be considered "safe" and Photoshop would be trusted to sign the resulting jpegs.
Like I said, I don't think it's practical...
Re: A new way to detect ‘deepfake’ picture editing
#90Earlier quoted context omitted.
I guess there'd have to be certified editors, which would be trusted to allow only "safe" operations on images (crop, white balance, levels, etc) and would sign the resulting jpegs themselves alongside the signature from the camera. It doesn't really seem likely at all to work in practice.
I want to see an ai that changes picture A to picture B by abusing rounding errors in white balance and levels adjustments.
I think the solution is for the editor to have a hard limit on how many edits it will apply to one image. You'd need to write into the output file how many edits had been performed before signing it and then refuse to process the edited image any further if it's too high.
Any scheme like this would be eventually broken somehow, so it's altogether a bad idea imho