Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Researchers tested AI watermarks and broke all of them

wired.com

Post reply on HN
Print view View original

71–80 of 91 posts

Re: Researchers tested AI watermarks and broke all of them

#71

Post by flextheruler »

j0hnyl wrote: post #70
jacobr1 wrote: post #10

We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…

The problem is that this requires a federated digital identity provider which is kind of the antithesis of a utopian internet.

Would it really be that different than https? I would assume it would mainly be large entities like governments or corporations utilizing these certificate chains

Re: Researchers tested AI watermarks and broke all of them

#72

Post by danShumway »

atestu wrote: post #54
nonameiguess wrote: post #50

Earlier quoted context omitted.

What's the threat vector you're trying to mitigate here? If you're wondering whether a movie that claims to be produced by Disney really was, if it's in theaters or on Disney+, then you can trust it was actually made by Disney or at least licensed to them. As long as the Washington Post still employs its own photographers and doesn't accept imagery submission from the general public, you should be able to trust a pho…

Well, a lot of people don't trust the Washington Post and might want proof that the photos they're posting are of real events. The Washington Post might not trust its photographers completely either (journalists making stuff up happens[0]), so they too might want proof the photos they're getting are real. [0]: https://www.nytimes.com/2003/05/11/us/correcting-the-record-...

I don't know; are you imagining a world where the people who believe the Washington Post would purposefully fabricate photos will be convinced by running a signature check on those photos?

Also bear in mind that Photoshop does exist today, and even without AI it would be well within the budget of the Washington Post to do edits to photographs that would be good enough to convince most non-experts. I don't run into many people saying their photos are doctored, but the conspiracy theories around photos today don't seem to be swayed by saying, "experts say it's not doctored."

I think if a chain of trust requires ordinary people to check signatures, it's probably not going to matter very much? I've seen people comparing this to HTTPS; people don't check HTTPS certificates either. There's a reason why PGP and signed messages haven't taken off. And the biggest criticism of Matrix that I see today when I introduce new users is that identity verification is too hard and they don't care about it. And these are the reasonable people.

I'm trying to imagine talking to someone who believes the mainstream media is lying about everything and telling them that, "no, it's OK, Google the tech company that you hate and that you think is trying to swing the election checked some math from its hardware and it says the photo is real." I don't think that's going to persuade those people of anything.

Re: Researchers tested AI watermarks and broke all of them

#73

Post by danShumway »

jacobr1 wrote: post #66
nonameiguess wrote: post #50

Earlier quoted context omitted.

What's the threat vector you're trying to mitigate here? If you're wondering whether a movie that claims to be produced by Disney really was, if it's in theaters or on Disney+, then you can trust it was actually made by Disney or at least licensed to them. As long as the Washington Post still employs its own photographers and doesn't accept imagery submission from the general public, you should be able to trust a pho…

Well, if I see a clip on youtube claiming to be from Disney ... it really might not be. If I see a photo on twitter claiming to come from the Washington Post, it might not be. If I see a photo in my facebook feed of a rioter, did it come from poster, or are they just reposting something else? Did that repost come from a newsource I trust, like the WP in this case, or from some reddit post, maybe edited or synthetical…

All of those are current problems though. Signing media isn't hard, determining who is trustworthy and how to do identity is hard. This the HTTPS problem; you can show that the connection is secure (assuming people care enough to check in the first place), but you still have to check if the connection isn't actually to Target and is to Torget instead.

Signing only helps if we have a reliable way to verify identities out of band, and we don't, that's one of the reasons the other problems you mention still exist today. Today, how do you determine that an article is actually from the Washington Post? You check the website. It's not like verification of whether an article exists is hard today -- it's everything else around it that's hard.

If checking if an article is actually on a website is too hard for people, importing a signing key is also going to be too hard. If people are confused remembering the Washington Post's URL, they won't learn how to use a signing key to check identity. Now maybe a website could automate that and put some kind of verification badge next to trustworthy verified identities, but I'm skeptical because Facebook did try to do that with news sources and it was a disaster and a bunch of politicians accused them of censorship.

We have a lot of mechanisms for verifying identity and sources of information that aren't leveraged today, and I think the immediate question to ask about a chain of trust is "what's going to make this different from all of the other chains of trust that people are already ignoring today?"

Re: Researchers tested AI watermarks and broke all of them

#74

Post by AnthonyMouse »

blibble wrote: post #41
tshaddox wrote: post #38

Earlier quoted context omitted.

That doesn't quite sound right to me. I don't think a new Blu-ray disc could be released that continues to be readable by some old readers but is no longer readable by other old readers.

> I don't think a new Blu-ray disc could be released that continues to be readable by some old readers but is no longer readable by other old readers. you can obviously think whatever you want, but you'd be completely wrong DVD supported this 20 years ago, blu-ray's system is far more sophisticated and can even block individual players The approach of AACS provisions each individual player with a unique set of decryp…

The problem, of course, being that some players will just read the raw bytes from the disc without even attempting to decrypt them, and then anyone can decrypt them in software using any other keys even if the player used to read the disc was revoked.

Then every time another player's keys are published it allows anyone to use the older player to read discs using the newer player's leaked keys. And some players are cracked but the keys aren't published, instead they use them to extract the disc key for every new disc and then publish all the disc keys, which can be used in the same way without revealing which player was cracked.

Re: Researchers tested AI watermarks and broke all of them

#75

Post by AnthonyMouse »

blibble wrote: post #52
floren wrote: post #17

Earlier quoted context omitted.

I was thinking the other day about embedding keys in cameras, etc. but came up with the problem that you could just wire up a computer that BEHAVES like a CCD sensor and send whatever the hell you feel like in to the signing hardware, so you feed in your fake image and it gets signed by the camera as though it were real. I assume smarter people than me have put much more time into the problem, so I'd be interested to…

you'd have to do the signing inside the ccd silicon (though if you have lots of time/effort/money you could still extract the key)

Or attach the inputs on the CCD silicon to something other than the CCD array.

Re: Researchers tested AI watermarks and broke all of them

#76

Post by AnthonyMouse »

tornato7 wrote: post #59
not2b wrote: post #57

Earlier quoted context omitted.

I could take a photo of someone else's photo with a camera that cryptographically signs the image. Then I suppose I could claim that my photo is the original (see? it is signed, with a camera that maintains a chain of trust) and the original photo is now the stolen one. To pull this off it would have to be a really high quality camera that would make an accurate copy. Perhaps something like this is what your hostile…

But if both images had a timestamp in the signature, wouldn’t you be able to prove that the original was taken first?

How does the device know what time it is?

What happens if the original was taken with any existing device that doesn't make signatures, or is a model that subsequently had its keys revoked?

Re: Researchers tested AI watermarks and broke all of them

#77

Post by AnthonyMouse »

flextheruler wrote: post #71
j0hnyl wrote: post #70

Earlier quoted context omitted.

The problem is that this requires a federated digital identity provider which is kind of the antithesis of a utopian internet.

Would it really be that different than https? I would assume it would mainly be large entities like governments or corporations utilizing these certificate chains

The certificates for HTTPS certify that you're communicating with someone the CA verified has control over some domain name. They don't require the CA to exercise any control over the endpoint device or for the user to cede any.

If you want the camera sensor in your phone to certify that the user hasn't altered the image, the device can't be in the control of the user.

Re: Researchers tested AI watermarks and broke all of them

#78

Post by AnthonyMouse »

danShumway wrote: post #68
Alligaturtle wrote: post #28

Earlier quoted context omitted.

I agree with this sentiment. Years ago, I asked around at one of the smartphone companies whether it would be possible to certify to an end user that a photo is either: 1) Authentic and only lightly edited with image manipulation software (e.g., cropped, color balanced, or text placed over top of the image) 2) Produced on a phone that has had to go through hardware hacks Note that the guarantee in (1) wouldn't preven…

> My favorite hostile response was that allowing people to certify images as real would steal peoples' rights. I didn't follow the logic, but the guy who made the argument was really into it. My guess is likely because it seems like this would be impossible to implement without adding DRM to the smartphone and/or locking down Open Source image editors out of the attestation process. You would need to prevent access t…

> mostly just because devs seem to have given up the fight about custom firmware on a phone in general

This is more like prioritization. If you can't even install your own apps, you focus on the gorilla holding a knife to your throat.

> But if you're talking about the phone signing the image after light editing like a crop has happened, at that point you're talking about moving this signature into user-space code, and while I'm sure that problem could have been explained better to you by the devs, it's not surprising to me at all that you'd get a hostile response to that suggestion because I don't see how it would be possible to do that without locking down user-space code.

That's the part that isn't a problem. If you had an existing image with an existing signature, you could modify it and store the changes as a diff against the original. You don't need or even want to sign it again, you just keep the original and its signature intact. Compressing two images that are nearly identical against each other shouldn't even have particularly high overhead.

Doing it this way would also be more secure because you wouldn't have to trust the device doing the modifications in any way.

The problem continues to be how to create such a signature to begin with, without depriving the user of control over their own property or leaving the keys inside of devices that are in the physical possession of every attacker in the world.

Re: Researchers tested AI watermarks and broke all of them

#79

Post by tornato7 »

AnthonyMouse wrote: post #76
tornato7 wrote: post #59

Earlier quoted context omitted.

But if both images had a timestamp in the signature, wouldn’t you be able to prove that the original was taken first?

How does the device know what time it is? What happens if the original was taken with any existing device that doesn't make signatures, or is a model that subsequently had its keys revoked?

You can store the hash on a public Blockchain and the timestamp of that transaction will be verifiable.

Re: Researchers tested AI watermarks and broke all of them

#80

Post by AnthonyMouse »

tornato7 wrote: post #79
AnthonyMouse wrote: post #76

Earlier quoted context omitted.

How does the device know what time it is? What happens if the original was taken with any existing device that doesn't make signatures, or is a model that subsequently had its keys revoked?

You can store the hash on a public Blockchain and the timestamp of that transaction will be verifiable.

If everybody does that with everything, blockchains get infeasibly large. If not, anybody can go and register things on a blockchain that the original creator didn't and then claim they were first.
Post reply on HN