Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SynthID: A tool to watermark and identify content generated through AI

deepmind.google

Post reply on HN
Print view View original

11–20 of 81 posts

Re: SynthID: A tool to watermark and identify content generated through AI

#11

Post by sippeangelo »

throwaway13337 wrote: post #5

These sorts of tools will only be able to positively identify a subset of genAI content. But I suspect that people will use it to 'prove' something is not genAI. In a sense, the identifier company can be an arbiter of the truth. Powerful. Training people on a half-solution like this might do more harm than good.

It is actively harmful to society. Slap SynthID on some of the photographic evidence from the unreleased Epstein files and instantly de-legitimize it. Launder a SynthID image through a watermark free model and it's legit again. The fact that it exists at all can't be interpreted in any other way than malice.

Re: SynthID: A tool to watermark and identify content generated through AI

#12

Post by observationist »

throwaway13337 wrote: post #5

These sorts of tools will only be able to positively identify a subset of genAI content. But I suspect that people will use it to 'prove' something is not genAI. In a sense, the identifier company can be an arbiter of the truth. Powerful. Training people on a half-solution like this might do more harm than good.

You could take a picture or video with your phone of a screen or projection of an altered media and thereby capture a watermarked "verified" image or video.

None of these schemes for validation of digital media will work. You need a web of trust, repeated trustworthy behavior by an actor demonstrating fidelity.

You need people and institutions you can trust, who have the capability of slogging through the ever more turbulent and murky sea of slop and using correlating evidence and scientific skepticism and all the cognitive tools available to get at reality. Such people and institutions exist. You can also successfully proxy validation of sources by identifying people or groups good at identifying primary sources.

When people and institutions defect, as many legacy media, platforms, talking heads, and others have, you need to ruthlessly cut them out of your information feed. When or if they correct their mistake, just follow tit for tat, and perhaps they can eventually earn back their place in the de-facto web of trust.

Google's stamp of approval means less than nothing to me; it's a countersignal, indicating I need to put even more effort than otherwise to confirm the truthfulness of any claims accompanied by their watermark.

Re: SynthID: A tool to watermark and identify content generated through AI

#13

Post by throwaway13337 »

greensoap wrote: post #8
throwaway13337 wrote: post #5

These sorts of tools will only be able to positively identify a subset of genAI content. But I suspect that people will use it to 'prove' something is not genAI. In a sense, the identifier company can be an arbiter of the truth. Powerful. Training people on a half-solution like this might do more harm than good.

It will just be an arms race if we try to prove "not genAI." Detectors will improve, genAI will improve without marking (opensource and state actors will have unmarked genAI even if we mandate it). Marking real from lense through digital life is more practical. But then what do we do with all the existing hardware that doesn't mark real and media that preexisited this problem.

I agree. A mechanism to voluntarily attach a certificate metadata about the media record from the device seems like a better idea. That still can be spoofed, though.

In the end, society has always existed on human chains of trust. Community. As long as there are human societies, we need human reputation.

Re: SynthID: A tool to watermark and identify content generated through AI

#14

Post by amingilani »

andrewmcwatters wrote: post #4

I wonder how it stands up to feature analysis. "Generate a pure white image." "Generate a pure black image." Channel diff, extract steganographic signature for analysis.

I just tried this idea, and it looks like it isn't that simple.

> "Generate a pure white image."

It refused no matter how I phrased it ¯\_(ツ)_/¯

> "Generate a pure black image."

It did give me one. In a new chat, I asked Gemini to detect SynthID with "@synthid". It responded with:

> The image contains too little information to make a diagnosis regarding whether it was created with Google AI. It is primarily a solid black field, and such content typically lacks the necessary data for SynthID to provide a definitive result.

Further research: Does a gradient trigger SynthID? IDK, I have to get back to work.

Re: SynthID: A tool to watermark and identify content generated through AI

#16

Post by gumby271 »

gregorkas wrote: post #6

I genuinely feel that in this AI world we need the inverse. That every analogue or digital photo taken by traditional means of photography will need to be signed by a certificate, so anyone can verify its authenticity.

I'm sure Apple would love that too. More seriously, would that also mean all editing tools would need to re-sign a photo that was previously signed by the original sensor. How do we distinguish an edit that's misleading vs just changing levels? It's an interesting area for sure, but this inverse approach seems much trickier.

Re: SynthID: A tool to watermark and identify content generated through AI

#17

Post by kingstnap »

It's security through obscurity. I'm sure with the technical details or even just sufficient access to a predictive oracle you could break this.

But I suppose it ads friction so better than nothing.

Watermarking text without affecting it is an interesting seemingly weird idea. Does it work any better than (with knowledge of the model used to produce said text), just observing the perplexity is low because its "on policy" generated text.

Re: SynthID: A tool to watermark and identify content generated through AI

#18

Post by graypegg »

hedora wrote: post #9
gregorkas wrote: post #6

I genuinely feel that in this AI world we need the inverse. That every analogue or digital photo taken by traditional means of photography will need to be signed by a certificate, so anyone can verify its authenticity.

Some cameras support this, but usually only for raw. Note that your cell phone camera is using gen AI techniques to counteract sensor noise. Was that famous person in the background really there, or a hallucination filling in static? Who knows at this point? So, the signatures you proposed need to have some nuance around what they’re asserting.

To be fair, I think just signing details about the way an image was assembled makes sense. Deciding on fake vs real doesn't have to be done at time of capture. We store things like the aperture size, sensitivity, camera name/model, etc in the EXIF data, including details about the image processing pipeline seems like a logical step. (With a signature verification scheme... and I guess also trying to embed that in the actual bitmap data)

There is no original image to recover, since we can't capture and describe every photon, so it's not a "fake vs real" image signature... that would be a UI choice the image viewer client would make based on the pipeline data in the image.

Re: SynthID: A tool to watermark and identify content generated through AI

#19

Post by yjftsjthsd-h »

gregorkas wrote: post #6

I genuinely feel that in this AI world we need the inverse. That every analogue or digital photo taken by traditional means of photography will need to be signed by a certificate, so anyone can verify its authenticity.

And how do you fix the analog hole? Because if you can point your "verified" camera at a sufficiently high-resolution screen, we're worse off than when we started.

Re: SynthID: A tool to watermark and identify content generated through AI

#20

Post by Coeur »

gregorkas wrote: post #6

I genuinely feel that in this AI world we need the inverse. That every analogue or digital photo taken by traditional means of photography will need to be signed by a certificate, so anyone can verify its authenticity.

This already exists: https://c2pa.org , https://en.wikipedia.org/wiki/Content_Authenticity_Initiativ... . Support by camera makers is - spotty.
Post reply on HN