Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Researchers tested AI watermarks and broke all of them

wired.com

Post reply on HN
Print view View original

51–60 of 91 posts

Re: Researchers tested AI watermarks and broke all of them

#51

Post by p1necone »

ryukoposting wrote: post #39
jacobr1 wrote: post #10

We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…

I was thinking the same thing- we've had digital signing algorithms for decades, and those seem to work fine enough. There's a healthy distrust of cert authorities, sure, but it still works .

The signature only proves that the website/Bill the photographer was involved in the chain sending the website/photograph to you, not anything about the content itself.

Unless you have reason to trust Bill himself you can't trust that he actually took the photo, or that it isn't ai generated. Although knowing that Bill isn't tech savvy enough to do those things might be enough.

Re: Researchers tested AI watermarks and broke all of them

#52

Post by blibble »

floren wrote: post #17
jacobr1 wrote: post #10

We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…

I was thinking the other day about embedding keys in cameras, etc. but came up with the problem that you could just wire up a computer that BEHAVES like a CCD sensor and send whatever the hell you feel like in to the signing hardware, so you feed in your fake image and it gets signed by the camera as though it were real. I assume smarter people than me have put much more time into the problem, so I'd be interested to…

you'd have to do the signing inside the ccd silicon

(though if you have lots of time/effort/money you could still extract the key)

Re: Researchers tested AI watermarks and broke all of them

#53

Post by atestu »

floren wrote: post #17
jacobr1 wrote: post #10

We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…

I was thinking the other day about embedding keys in cameras, etc. but came up with the problem that you could just wire up a computer that BEHAVES like a CCD sensor and send whatever the hell you feel like in to the signing hardware, so you feed in your fake image and it gets signed by the camera as though it were real. I assume smarter people than me have put much more time into the problem, so I'd be interested to…

What about adding other sensors like LIDAR to make it harder to fake what you’re filming? I think about it often too…

Re: Researchers tested AI watermarks and broke all of them

#54

Post by atestu »

nonameiguess wrote: post #50
jacobr1 wrote: post #10

We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…

What's the threat vector you're trying to mitigate here? If you're wondering whether a movie that claims to be produced by Disney really was, if it's in theaters or on Disney+, then you can trust it was actually made by Disney or at least licensed to them. As long as the Washington Post still employs its own photographers and doesn't accept imagery submission from the general public, you should be able to trust a pho…

Well, a lot of people don't trust the Washington Post and might want proof that the photos they're posting are of real events.

The Washington Post might not trust its photographers completely either (journalists making stuff up happens[0]), so they too might want proof the photos they're getting are real.

[0]: https://www.nytimes.com/2003/05/11/us/correcting-the-record-...

Re: Researchers tested AI watermarks and broke all of them

#55

Post by jallmann »

jacobr1 wrote: post #10

We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…

Agreed that cryptographic provenance seems like the best way forward, however the goal is more about proving authenticity, less about detecting fraud. Small but important distinction.

There are industry initiatives around this already such as CAI https://en.m.wikipedia.org/wiki/Content_Authenticity_Initiat...

Re: Researchers tested AI watermarks and broke all of them

#57

Post by not2b »

Alligaturtle wrote: post #28
jacobr1 wrote: post #10

We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…

I agree with this sentiment. Years ago, I asked around at one of the smartphone companies whether it would be possible to certify to an end user that a photo is either: 1) Authentic and only lightly edited with image manipulation software (e.g., cropped, color balanced, or text placed over top of the image) 2) Produced on a phone that has had to go through hardware hacks Note that the guarantee in (1) wouldn't preven…

I could take a photo of someone else's photo with a camera that cryptographically signs the image. Then I suppose I could claim that my photo is the original (see? it is signed, with a camera that maintains a chain of trust) and the original photo is now the stolen one. To pull this off it would have to be a really high quality camera that would make an accurate copy.

Perhaps something like this is what your hostile responder was thinking of.

Re: Researchers tested AI watermarks and broke all of them

#58

Post by tornato7 »

yetanotherloss wrote: post #18
jacobr1 wrote: post #10

We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…

The cryptography to support this has been around for ever and it's been next to impossible to make the decision makers at companies and large organizations care, much less end users. Small time players like GE routinely fail to correctly sign industrial control software, the odds of people recording video paying enough attention to get it right and the meme crowd bothering to check even if they did seems vanishingly…

For practical uses the cryptography is quite new. Essentially if you cryptographically sign an image coming out of a sensor then you’re only getting whatever original version of it is and if you modify it even just to resize it for web for example, then it’s going to break the cryptographic chain of trust. You need to use zero knowledge proofs to allow for necessary image manipulations while keeping a signature.

Re: Researchers tested AI watermarks and broke all of them

#59

Post by tornato7 »

not2b wrote: post #57
Alligaturtle wrote: post #28

Earlier quoted context omitted.

I agree with this sentiment. Years ago, I asked around at one of the smartphone companies whether it would be possible to certify to an end user that a photo is either: 1) Authentic and only lightly edited with image manipulation software (e.g., cropped, color balanced, or text placed over top of the image) 2) Produced on a phone that has had to go through hardware hacks Note that the guarantee in (1) wouldn't preven…

I could take a photo of someone else's photo with a camera that cryptographically signs the image. Then I suppose I could claim that my photo is the original (see? it is signed, with a camera that maintains a chain of trust) and the original photo is now the stolen one. To pull this off it would have to be a really high quality camera that would make an accurate copy. Perhaps something like this is what your hostile…

But if both images had a timestamp in the signature, wouldn’t you be able to prove that the original was taken first?

Re: Researchers tested AI watermarks and broke all of them

#60

Post by tornato7 »

nofunphil wrote: post #23
jacobr1 wrote: post #10

We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…

Agreed. At the risk of a shitstorm of downvotes, tokenized media could be part of a solution, especially at the consumer level. Authenticate real videos via a mint button/QR that airdrops you a token from creator. May require platforms to opt-in tho. Basically trust nothing unless you can authenticate source onchain. Not great fo sho, but prob necessary soon

This is the best solution IMO - Blockchain allows others to independently verify the time that media was created without needing to trust some random company's database / API in perpetuity.
Post reply on HN