Researchers tested AI watermarks and broke all of them • Hacker News Board

Re: Researchers tested AI watermarks and broke all of them

#81

Post by jallmann » Sat, Oct 07, 2023, 6:30 AM UTC

jacobr1 wrote: post #64

jallmann wrote: post #55

Earlier quoted context omitted.

Agreed that cryptographic provenance seems like the best way forward, however the goal is more about proving authenticity, less about detecting fraud. Small but important distinction. There are industry initiatives around this already such as CAI https://en.m.wikipedia.org/wiki/Content_Authenticity_Initiat...

CAI is roughly the kind of scheme I was suggesting. Thanks for the reference. My take is that proving authenticity might not be something we can do with any degree of accuracy in a general sense. So if that is infeasible, then we need _some_ kind of mitigation. Something like CAI allows us to make the an assessment about the how much trust to give an informational source, probably taking into account multiple factors…

Exactly - content authenticity is a narrower and more tractable problem.

And I suspect in most cases that is enough - the world runs on trust, so if a reliable source (eg, the NY Times) attests that a photograph is authentic, then we can reasonably trust that. And as you noted, this chain of provenance can go all the way back to the device itself.

Fraud, on the other hand, is much harder to prove without a doubt. It is still a problem, but probably less so in the general case [1]. The concerning thing seems to be extremely targeted attacks, eg, hacking a CCTV to implicate someone in a crime.

[1] Notwithstanding folks who slurp up unfiltered content on, eg, TikTok, but the most outrageous stuff usually doesn't "make the news" on its own without more vetting. If anything, this will reinforce the importance of actual journalism, fact-checking and corroborating evidence.

Re: Researchers tested AI watermarks and broke all of them

#82

Post by tornato7 » Sat, Oct 07, 2023, 8:09 AM UTC

AnthonyMouse wrote: post #80

tornato7 wrote: post #79

Earlier quoted context omitted.

You can store the hash on a public Blockchain and the timestamp of that transaction will be verifiable.

If everybody does that with everything, blockchains get infeasibly large. If not, anybody can go and register things on a blockchain that the original creator didn't and then claim they were first.

You would just use any sort of aggregation scheme to include multiple hashes at once. Even concatenating 1000 image hashes and hashing that would allow you to prove later that they were all included.

Re: Researchers tested AI watermarks and broke all of them

#83

Post by whywhywhywhy » Sat, Oct 07, 2023, 11:01 AM UTC

I’ll never get over the “invisible_watermark” Python package being entirely visible to the naked eye, obviously degrades the image in an way that’s unacceptable and even easily spottable on any image once you know what it looks like.

Re: Researchers tested AI watermarks and broke all of them

#84

Post by danShumway » Sat, Oct 07, 2023, 1:11 PM UTC

AnthonyMouse wrote: post #78

danShumway wrote: post #68

Earlier quoted context omitted.

> My favorite hostile response was that allowing people to certify images as real would steal peoples' rights. I didn't follow the logic, but the guy who made the argument was really into it. My guess is likely because it seems like this would be impossible to implement without adding DRM to the smartphone and/or locking down Open Source image editors out of the attestation process. You would need to prevent access t…

> mostly just because devs seem to have given up the fight about custom firmware on a phone in general This is more like prioritization. If you can't even install your own apps, you focus on the gorilla holding a knife to your throat. > But if you're talking about the phone signing the image after light editing like a crop has happened, at that point you're talking about moving this signature into user-space code, an…

> That's the part that isn't a problem. If you had an existing image with an existing signature, you could modify it and store the changes as a diff against the original.

I think this glosses over things a little bit. Are you going to transmit the original and the diff to every image viewer? People are talking about doing these checks on clientside devices, not just having an attestation check somewhere else.

Ultimately the only way you can check this is to give someone the original and the signature to compare. Want to blur or censor a face? Tough. Want to crop? Tough. And the person doing that verification would want to be able to look at the photograph to tell how extensive your edits were.

Technically what you're saying is true in that you could do diffs this way, but in practice you'd have to commit to publishing the pre-edit photo. We're also suddenly no longer talking about a behind-the-scenes process that just puts a little green check on the photo or something; because edits can be anything and only the original photo would be signed; so the "verification" in your image editor would now be a software stack that shows you the original photo alongside the edits I guess?

----

> The problem continues to be how to create such a signature to begin with, without depriving the user of control over their own property or leaving the keys inside of devices that are in the physical possession of every attacker in the world.

I'm quibbling though, I think we're mostly in agreement. This is the DRM aspect that people seem to be forgetting. Commentary about attestation is not making the obvious and direct comparison that controlling device behavior is already something companies are trying to do and failing at.

Re: Researchers tested AI watermarks and broke all of them

#85

Post by tshaddox » Sat, Oct 07, 2023, 3:23 PM UTC

blibble wrote: post #41

tshaddox wrote: post #38

Earlier quoted context omitted.

That doesn't quite sound right to me. I don't think a new Blu-ray disc could be released that continues to be readable by some old readers but is no longer readable by other old readers.

> I don't think a new Blu-ray disc could be released that continues to be readable by some old readers but is no longer readable by other old readers. you can obviously think whatever you want, but you'd be completely wrong DVD supported this 20 years ago, blu-ray's system is far more sophisticated and can even block individual players The approach of AACS provisions each individual player with a unique set of decryp…

For CSS and AACS, yes. I was referring specifically to HDCP, which involves negotiation between source and sink devices and AFAIK has nothing like broadcast encryption.

Re: Researchers tested AI watermarks and broke all of them

#86

Post by AnthonyMouse » Sat, Oct 07, 2023, 3:57 PM UTC

danShumway wrote: post #84

AnthonyMouse wrote: post #78

Earlier quoted context omitted.

> mostly just because devs seem to have given up the fight about custom firmware on a phone in general This is more like prioritization. If you can't even install your own apps, you focus on the gorilla holding a knife to your throat. > But if you're talking about the phone signing the image after light editing like a crop has happened, at that point you're talking about moving this signature into user-space code, an…

> That's the part that isn't a problem. If you had an existing image with an existing signature, you could modify it and store the changes as a diff against the original. I think this glosses over things a little bit. Are you going to transmit the original and the diff to every image viewer? People are talking about doing these checks on clientside devices, not just having an attestation check somewhere else. Ultimat…

> Ultimately the only way you can check this is to give someone the original and the signature to compare. Want to blur or censor a face? Tough. Want to crop? Tough. And the person doing that verification would want to be able to look at the photograph to tell how extensive your edits were.

Well of course they would. Otherwise what are you even trying to attest? Otherwise someone could take an image from a camera, replace literally every pixel with whatever they want and then claim it's the same image.

Re: Researchers tested AI watermarks and broke all of them

#87

Post by danShumway » Sat, Oct 07, 2023, 4:06 PM UTC

AnthonyMouse wrote: post #86

danShumway wrote: post #84

Earlier quoted context omitted.

> That's the part that isn't a problem. If you had an existing image with an existing signature, you could modify it and store the changes as a diff against the original. I think this glosses over things a little bit. Are you going to transmit the original and the diff to every image viewer? People are talking about doing these checks on clientside devices, not just having an attestation check somewhere else. Ultimat…

> Ultimately the only way you can check this is to give someone the original and the signature to compare. Want to blur or censor a face? Tough. Want to crop? Tough. And the person doing that verification would want to be able to look at the photograph to tell how extensive your edits were. Well of course they would. Otherwise what are you even trying to attest? Otherwise someone could take an image from a camera, re…

Agreed, but at that point, why have the edits anymore; particularly if you expect people to actually check.

Obscuring faces for privacy, cropping would no longer work. And even minor touchups like lighting would be of questionable value since you're expecting users not to look at the edited photo or at least to primarily look at the edited photo next to the original.

I suspect in practice that doing edits on top of a signed photo would be basically the same as not having editing capability at all; and even that's assuming users would compare the edited and non-edited versions at all, which is not a safe assumption in my mind given how hard it is to even get people to click into a full article past the headline.

Re: Researchers tested AI watermarks and broke all of them

#88

Post by AnthonyMouse » Sat, Oct 07, 2023, 4:07 PM UTC

tornato7 wrote: post #82

AnthonyMouse wrote: post #80

Earlier quoted context omitted.

If everybody does that with everything, blockchains get infeasibly large. If not, anybody can go and register things on a blockchain that the original creator didn't and then claim they were first.

You would just use any sort of aggregation scheme to include multiple hashes at once. Even concatenating 1000 image hashes and hashing that would allow you to prove later that they were all included.

That just moves the problem from where to store the blockchain to where to store the concatenated hashes.

If this is some cloud provider, what are you getting from a blockchain? Just have the cloud provider do the certification. If they betray you or go out of business you've lost your hashes anyway.

If it's stored on the endpoint device, you can't prove it anymore if the device gets lost or damaged. In theory people could back them up, but we all know perfectly well that ordinary people are not going to do that unless it's automated.

So then you're back to storing them in a distributed system, i.e. making them a necessary part of the blockchain. And then it gets too big.

Re: Researchers tested AI watermarks and broke all of them

#89

Post by AnthonyMouse » Sat, Oct 07, 2023, 5:58 PM UTC

danShumway wrote: post #87

AnthonyMouse wrote: post #86

Earlier quoted context omitted.

> Ultimately the only way you can check this is to give someone the original and the signature to compare. Want to blur or censor a face? Tough. Want to crop? Tough. And the person doing that verification would want to be able to look at the photograph to tell how extensive your edits were. Well of course they would. Otherwise what are you even trying to attest? Otherwise someone could take an image from a camera, re…

Agreed, but at that point, why have the edits anymore; particularly if you expect people to actually check. Obscuring faces for privacy, cropping would no longer work. And even minor touchups like lighting would be of questionable value since you're expecting users not to look at the edited photo or at least to primarily look at the edited photo next to the original. I suspect in practice that doing edits on top of a…

The value isn't that the users are going to do it under normal circumstances. They would see the edited photo. You'd only care about the signature if its provenance came into question.

You could also handle cropping and omissions by having the original device sign the picture as a grid of individual tiles. Then you could omit some and still prove that the others are original.

Re: Researchers tested AI watermarks and broke all of them

#90

Post by danShumway » Sat, Oct 07, 2023, 7:59 PM UTC

AnthonyMouse wrote: post #89

danShumway wrote: post #87

Earlier quoted context omitted.

Agreed, but at that point, why have the edits anymore; particularly if you expect people to actually check. Obscuring faces for privacy, cropping would no longer work. And even minor touchups like lighting would be of questionable value since you're expecting users not to look at the edited photo or at least to primarily look at the edited photo next to the original. I suspect in practice that doing edits on top of a…

The value isn't that the users are going to do it under normal circumstances. They would see the edited photo. You'd only care about the signature if its provenance came into question. You could also handle cropping and omissions by having the original device sign the picture as a grid of individual tiles. Then you could omit some and still prove that the others are original.

I'm still skeptical that this would end up working well in practice, but I do want to say:

> You could also handle cropping and omissions by having the original device sign the picture as a grid of individual tiles.

is a pretty good idea, I like that quite a lot. Not saying it means I'm on board with signatures overall (I mean, we're still in agreement that this would require locking down devices to at least some degree) but I do think that's an elegant solution for the cropping/censoring part of it.