A new way to detect ‘deepfake’ picture editing • Hacker News Board

Re: A new way to detect ‘deepfake’ picture editing

#61

Post by arbitrage » Fri, Sep 03, 2021, 1:55 PM UTC

sigmoid10 wrote: post #36

Etheryte wrote: post #26

It's hard to see this as much anything but snake oil. While the technique allegedly allows you to target multiple models, it still requires you to know what adversarial model you're targeting before the fact. If the adversary sees there's visible artifacts they'll just change a few parameters or use a different model and be good to go none the less. In short, while this might be academically interesting, I can't help…

A few weeks ago, there was a post here about that guy who deepfaked female profile pictures. He showed that the visible artifacts go away by merely scaling up the number of parameters in the model. Give it a few years and these fake images will be impossible to detect unless we agree on some way of cryptographically signing images from real cameras.

define 'real camera', please.

Re: A new way to detect ‘deepfake’ picture editing

#62

Post by xrendan » Fri, Sep 03, 2021, 1:58 PM UTC

uuidgen wrote: post #57

sebzim4500 wrote: post #56

Earlier quoted context omitted.

>Ssl certificates have been facing this problem for years. That's an entirely different, and much easier problem. In the case of SSL you are not worried about an attacker obtaining a certificate for any host, just the ones you care about. E.g. an attacker getting a certificate for facebook.com would be catastrophic, but an attacker getting a certificate for a website that no one uses would be a non-issue. For the cas…

Every camera will have a unique key so a single key leaked is still no issue. It's easy to maintain a blacklist - similarly how CRLs work.

Yes, but you need to detect the fakes before you can ban them. An adversary could just rotate camera keys before they're even detected.

Re: A new way to detect ‘deepfake’ picture editing

#63

Post by smaryjerry » Fri, Sep 03, 2021, 2:06 PM UTC

sigmoid10 wrote: post #36

Etheryte wrote: post #26

It's hard to see this as much anything but snake oil. While the technique allegedly allows you to target multiple models, it still requires you to know what adversarial model you're targeting before the fact. If the adversary sees there's visible artifacts they'll just change a few parameters or use a different model and be good to go none the less. In short, while this might be academically interesting, I can't help…

A few weeks ago, there was a post here about that guy who deepfaked female profile pictures. He showed that the visible artifacts go away by merely scaling up the number of parameters in the model. Give it a few years and these fake images will be impossible to detect unless we agree on some way of cryptographically signing images from real cameras.

Imagine a cryptocurrency like log of data verifying images. Essentially, all cameras will be internet connected with a process to force creation of an NFT at the moment users press the photo button which includes the entire data of the image. Any image without that tech will be considered unreliable. Of course that would have implications that no photo could ever be deleted, I’m guessing this system would be government based and they would be just fine with that.

Re: A new way to detect ‘deepfake’ picture editing

#64

Post by spywaregorilla » Fri, Sep 03, 2021, 2:16 PM UTC

roywiggins wrote: post #60

spywaregorilla wrote: post #58

Earlier quoted context omitted.

At best that would allow you to tell if a picture was raw footage. Which is pointless because people edit things all the time.

I guess there'd have to be certified editors, which would be trusted to allow only "safe" operations on images (crop, white balance, levels, etc) and would sign the resulting jpegs themselves alongside the signature from the camera. It doesn't really seem likely at all to work in practice.

I want to see an ai that changes picture A to picture B by abusing rounding errors in white balance and levels adjustments.

Re: A new way to detect ‘deepfake’ picture editing

#65

Post by rspeele » Fri, Sep 03, 2021, 2:29 PM UTC

smaryjerry wrote: post #63

sigmoid10 wrote: post #36

Earlier quoted context omitted.

A few weeks ago, there was a post here about that guy who deepfaked female profile pictures. He showed that the visible artifacts go away by merely scaling up the number of parameters in the model. Give it a few years and these fake images will be impossible to detect unless we agree on some way of cryptographically signing images from real cameras.

Imagine a cryptocurrency like log of data verifying images. Essentially, all cameras will be internet connected with a process to force creation of an NFT at the moment users press the photo button which includes the entire data of the image. Any image without that tech will be considered unreliable. Of course that would have implications that no photo could ever be deleted, I’m guessing this system would be governme…

What can crypto verify here exactly? What stops me from:

A. Writing my own software "camera" that does the same gyrations a real camera would do to sign its raw image, but applied to an arbitrary unverified image file on my computer?

or

B. Printing out an unverified picture and taking a picture of that picture (with good lighting etc so that it is not obvious), with a camera that makes it verified "real"?

Re: A new way to detect ‘deepfake’ picture editing

#66

Post by rspeele » Fri, Sep 03, 2021, 2:31 PM UTC

sigmoid10 wrote: post #50

sideshowb wrote: post #39

Earlier quoted context omitted.

I'm not hopeful that the latter will do much. Just do your deepfake, point a camera at your screen and press the shutter button to get it cryptographically signed..? Ok, so that's tongue in cheek and we'd see some artifacts there but the general principle works. You could intercept the signal from the CCD, or just extract the signing key from the camera's ROM, etc etc.

You can already do that today to fool basic image forensics. Security is never absolute, but if you make the hassle big enough it might suffice to protect the general population. Just like almost all smartphones don't let you mess with the wifi card or spoof your mac address unless you root them. If that wasn't the case, you'd see way more hostapd-wpe attacks these days, since every kid could do it.

In the case of image trustworthiness, it might actually be better for it to be accessible to the general population, if it's doable at all. Then the public knows that images are unreliable evidence.

Re: A new way to detect ‘deepfake’ picture editing

#67

Post by mzs » Fri, Sep 03, 2021, 2:49 PM UTC

dmos62 wrote: post #25

The post is pretty hard to understand. Here's the paper and an excerpt from the abstract: https://arxiv.org/abs/2106.00660 > First, we show how an image owner with access to an inpainting model can augment their image in such a way that any attempt to edit it using that model will add arbitrary visible information. We find that we can target multiple different models simultaneously with our technique. This can be des…

Thanks, this is the code with examples in the readme:

https://github.com/iliaishacked/markpainting

Re: A new way to detect ‘deepfake’ picture editing

#68

Post by Closi » Fri, Sep 03, 2021, 2:51 PM UTC

Etheryte wrote: post #26

It's hard to see this as much anything but snake oil. While the technique allegedly allows you to target multiple models, it still requires you to know what adversarial model you're targeting before the fact. If the adversary sees there's visible artifacts they'll just change a few parameters or use a different model and be good to go none the less. In short, while this might be academically interesting, I can't help…

This is fine, we will just train another model to detect which model is being targeted.

Then if there is an adversarial model built for that, we will just create a model to detect images that have been changed to hide the model they were built with.

Then we will build a model for the adversarial AI on top of that and so on.

Re: A new way to detect ‘deepfake’ picture editing

#69

Post by sigmoid10 » Fri, Sep 03, 2021, 3:08 PM UTC

xrendan wrote: post #62

uuidgen wrote: post #57

Earlier quoted context omitted.

Every camera will have a unique key so a single key leaked is still no issue. It's easy to maintain a blacklist - similarly how CRLs work.

Yes, but you need to detect the fakes before you can ban them. An adversary could just rotate camera keys before they're even detected.

You just need to couple the keys to something tangible. But the truth is that every system can by bypassed if you spend enough effort. The only question is when it becomes enough hassle so that it doesn't disrupt ordinary people's lives. Not even facebook cared about enforcing https until someone made a browser plugin that let everyone steal cookies.

Re: A new way to detect ‘deepfake’ picture editing

#70

Post by recursive » Fri, Sep 03, 2021, 3:33 PM UTC

rspeele wrote: post #65

smaryjerry wrote: post #63

Earlier quoted context omitted.

Imagine a cryptocurrency like log of data verifying images. Essentially, all cameras will be internet connected with a process to force creation of an NFT at the moment users press the photo button which includes the entire data of the image. Any image without that tech will be considered unreliable. Of course that would have implications that no photo could ever be deleted, I’m guessing this system would be governme…

What can crypto verify here exactly? What stops me from: A. Writing my own software "camera" that does the same gyrations a real camera would do to sign its raw image, but applied to an arbitrary unverified image file on my computer? or B. Printing out an unverified picture and taking a picture of that picture (with good lighting etc so that it is not obvious), with a camera that makes it verified "real"?

A. wouldn't work because there would presumably be a private key or something.

B. seems like it would work.