Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Researchers tested AI watermarks and broke all of them

wired.com

Post reply on HN
Print view View original

21–30 of 91 posts

Re: Researchers tested AI watermarks and broke all of them

#21

Post by tempusalaria »

jacobr1 wrote: post #10

We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…

You can see here: GitHub.com/HNx1/IdentityLM

It’s a direct (and open source) implementation of public key cryptography into the LLM logit distribution.

The paraphrasing model/beam search needs work - feel free to pitch in :)

Re: Researchers tested AI watermarks and broke all of them

#22

Post by tshaddox »

floren wrote: post #17
jacobr1 wrote: post #10

We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…

I was thinking the other day about embedding keys in cameras, etc. but came up with the problem that you could just wire up a computer that BEHAVES like a CCD sensor and send whatever the hell you feel like in to the signing hardware, so you feed in your fake image and it gets signed by the camera as though it were real. I assume smarter people than me have put much more time into the problem, so I'd be interested to…

I think you're essentially describing the hardware DRM supply chain.

For example, HDCP is a DRM scheme where Intel convinces (or legally requires) every manufacturer of HDMI output devices (e.g. set-top boxes, Blu-ray players) in the world to encrypt certain video streams.

Then, Intel requires manufacturers of HDMI input devices (e.g. TVs) to purchase a license key that can decrypt those video streams. This license agreement also requires the manufacturer to design their device such that the device key cannot be easily discovered and the video content cannot be easily copied.

Then, Intel gets media companies to include some extra metadata in video media like Blu-ray discs. This metadata can contain revoked device keys, so that if a TV manufacturer violates the terms of the license agreement (e.g. leaks their key, or sells a device that makes copies of video content), that manufacturer's TVs won't be able to play new content that starts including their key in the revocation list.

Of course, Intel's HDCP master key was either leaked or reverse-engineered, so anyone can generate their own valid device keys. Intel will probably sue you if you do this, I guess.

Re: Researchers tested AI watermarks and broke all of them

#23

Post by nofunphil »

jacobr1 wrote: post #10

We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…

Agreed. At the risk of a shitstorm of downvotes, tokenized media could be part of a solution, especially at the consumer level. Authenticate real videos via a mint button/QR that airdrops you a token from creator. May require platforms to opt-in tho. Basically trust nothing unless you can authenticate source onchain. Not great fo sho, but prob necessary soon

Re: Researchers tested AI watermarks and broke all of them

#24

Post by charcircuit »

tshaddox wrote: post #22
floren wrote: post #17

Earlier quoted context omitted.

I was thinking the other day about embedding keys in cameras, etc. but came up with the problem that you could just wire up a computer that BEHAVES like a CCD sensor and send whatever the hell you feel like in to the signing hardware, so you feed in your fake image and it gets signed by the camera as though it were real. I assume smarter people than me have put much more time into the problem, so I'd be interested to…

I think you're essentially describing the hardware DRM supply chain. For example, HDCP is a DRM scheme where Intel convinces (or legally requires) every manufacturer of HDMI output devices (e.g. set-top boxes, Blu-ray players) in the world to encrypt certain video streams. Then, Intel requires manufacturers of HDMI input devices (e.g. TVs) to purchase a license key that can decrypt those video streams. This license a…

>Of course, Intel's HDCP master key was either leaked or reverse-engineered, so anyone can generate their own valid device keys

Of an older version of HDCP. New media can require a higher HDCP version where that bypass isn't possible.

Re: Researchers tested AI watermarks and broke all of them

#25

Post by charcircuit »

epivosism wrote: post #16

Wasn't this obvious from the get go that this can't work? If AI will eventually generate say 10k by 10k images, I can resize to 2.001k by 1.999k or similar, and I just don't get how any subtle signal in the pixels can persist through that. Maybe you could do something at the compositional level, but that seems restrictive to the output. Maybe something about like larger regions average color balance or something? But…

Normal watermarking solutions can survive resizes.

Re: Researchers tested AI watermarks and broke all of them

#26

Post by sebzim4500 »

tshaddox wrote: post #22
floren wrote: post #17

Earlier quoted context omitted.

I was thinking the other day about embedding keys in cameras, etc. but came up with the problem that you could just wire up a computer that BEHAVES like a CCD sensor and send whatever the hell you feel like in to the signing hardware, so you feed in your fake image and it gets signed by the camera as though it were real. I assume smarter people than me have put much more time into the problem, so I'd be interested to…

I think you're essentially describing the hardware DRM supply chain. For example, HDCP is a DRM scheme where Intel convinces (or legally requires) every manufacturer of HDMI output devices (e.g. set-top boxes, Blu-ray players) in the world to encrypt certain video streams. Then, Intel requires manufacturers of HDMI input devices (e.g. TVs) to purchase a license key that can decrypt those video streams. This license a…

Interesting. I don't understand the revocation process though.

What stops the blu-ray reader from just ignoring the revocation list on the disk?

Re: Researchers tested AI watermarks and broke all of them

#28

Post by Alligaturtle »

jacobr1 wrote: post #10

We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…

I agree with this sentiment. Years ago, I asked around at one of the smartphone companies whether it would be possible to certify to an end user that a photo is either:

1) Authentic and only lightly edited with image manipulation software (e.g., cropped, color balanced, or text placed over top of the image) 2) Produced on a phone that has had to go through hardware hacks

Note that the guarantee in (1) wouldn't prevent someone from taking a photo of a TV screen. When I asked that original question, I had quite a few more details about how the certification might be done, how the credentials would be hosted, and how the results would be shown on a website.

Anyway, just asking this question was met with a storm of negative responses. I counted two dozen messages that were either neutral (asking for clarification) or else outright hostile before the first hesitantly positive message. My favorite hostile response was that allowing people to certify images as real would steal peoples' rights. I didn't follow the logic, but the guy who made the argument was really into it.

There were lots of comments about how using AI would be a better solution, some commenting on how Cannon already did it (and messed up gloriously), others stating they didn't have faith in hardware... it makes a fella never want to ask a question again.

In the end, I got an expert to speculate that the technology currently exists, and has existed for 5-10 years, to do this with a modern smartphone. However, unless a high-level engineer or executive argues that providing this feature will somehow be a competitive advantage, there is no appetite to provide this kind of feature.

Re: Researchers tested AI watermarks and broke all of them

#30

Post by tomrod »

yetanotherloss wrote: post #18
jacobr1 wrote: post #10

We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that c…

The cryptography to support this has been around for ever and it's been next to impossible to make the decision makers at companies and large organizations care, much less end users. Small time players like GE routinely fail to correctly sign industrial control software, the odds of people recording video paying enough attention to get it right and the meme crowd bothering to check even if they did seems vanishingly…

> Small time players like GE routinely fail to correctly sign industrial control software, the odds of people recording video paying enough attention to get it right and the meme crowd bothering to check even if they did seems vanishingly small without a lot of educational effort.

I've wanted to build a product in this space ever since I heard about deepfakes. Mix of keybase and appropriate file hash, and hash gen for subsets of sections of video. Maybe it needs to be a protocol, maybe a product, not sure, but the need seems apparent to me.

Post reply on HN