Live data from Hacker News

SynthID: A tool to watermark and identify content generated through AI

deepmind.google

61–70 of 81 posts

Re: SynthID: A tool to watermark and identify content generated through AI

#61
post #45

Earlier quoted context omitted.

Yeah, they also outlawed murder. And stealing. And bribing officials. All universally unenforceable. Weird...

Well, consider the case with murder: they're not demanding that people proactively implement a system to prevent it from happening, are they? You're just not allowed to do it, in the sense that the system will attempt to find you, prove your guilt, and punish you after the fact.

There are various systems meant to (attempt to) prevent it from happening, yes, from firearms laws to police forces

But picking out murder and ignoring the other ones which are far more analogous to the regulations mentioned seems a bit disingenuous...

Re: SynthID: A tool to watermark and identify content generated through AI

#62
post #20

I genuinely feel that in this AI world we need the inverse. That every analogue or digital photo taken by traditional means of photography will need to be signed by a certificate, so anyone can verify its authenticity.

This already exists: https://c2pa.org , https://en.wikipedia.org/wiki/Content_Authenticity_Initiativ... . Support by camera makers is - spotty.

Doesn't this require a paid certificate? that effectively blocks open source software/hardware from implementing it.

Re: SynthID: A tool to watermark and identify content generated through AI

#63

Earlier quoted context omitted.

You can trivially enforce that at the AI provider level, which covers 99% of the problem the law is designed to address. Of course it doesn't cover the issue of foreign state psyop operations but the fact that enforcing laws against organized crime and adversary state actors is hard isn't specific to AI.

Are you not aware of open-weights models and local generation? I think the vast majority of deepfake content is being genned in basements on RTX cards, not on public providers. People already have all this content, and have archives of it, and can run it airgapped. Cat is out of bag.

I am well aware of them, and I'm well aware that they are very niche as I'm the only one of my surrounding to use one of those. And those very models are being developed by tech giants and VC backed companies, on which regulation have leverage.

The fact that a small black market exists doesn't mean regulating the mainstream market doesn't matters.

Also, most people like you fail to realizes that the EU only has mandate from the member states to regulate the economy. The EU has no business dealing with people using SDXL finetunes on RTX cards in their garage.

Re: SynthID: A tool to watermark and identify content generated through AI

#65

Earlier quoted context omitted.

You can trivially enforce that at the AI provider level, which covers 99% of the problem the law is designed to address. Of course it doesn't cover the issue of foreign state psyop operations but the fact that enforcing laws against organized crime and adversary state actors is hard isn't specific to AI.

Are you not aware of open-weights models and local generation? I think the vast majority of deepfake content is being genned in basements on RTX cards, not on public providers. People already have all this content, and have archives of it, and can run it airgapped. Cat is out of bag.

I would be very surprised if that would be the case. Maybe you mean deepfake content generated by organized crime or state actors, but that surely is a tiny fraction of what's being generated on Grok or other platforms.

Re: SynthID: A tool to watermark and identify content generated through AI

#66

I genuinely feel that in this AI world we need the inverse. That every analogue or digital photo taken by traditional means of photography will need to be signed by a certificate, so anyone can verify its authenticity.

And how do you fix the analog hole? Because if you can point your "verified" camera at a sufficiently high-resolution screen, we're worse off than when we started.

You can have other sensors that tell you it's a screen, maybe require a Live Photo, maybe also upload to a third party service faster than generation is possible? In the end I think we'd end up somewhere like with cryptography: generating a real fake might be theoretically possible but it could be made prohibitively expensive to generate.

Re: SynthID: A tool to watermark and identify content generated through AI

#67
post #59

Earlier quoted context omitted.

> they're not demanding that people proactively implement a system to prevent it from happening, are they? What do you think a "background check" is?

Definitely not murder prevention

That's absolutely what they are. That and other crimes. That's why they're mandatory, by law, in certain industries. That's _precisely_ why we started using them: to prevent the easily preventable.

I suppose this logic stands in the way of a corporation getting what it wants and so it's automatically offensive to the HN "job seeking" crowd; however, even a basic reading of the history shows it's completely true.

Re: SynthID: A tool to watermark and identify content generated through AI

#68

I genuinely feel that in this AI world we need the inverse. That every analogue or digital photo taken by traditional means of photography will need to be signed by a certificate, so anyone can verify its authenticity.

I'm sure Apple would love that too. More seriously, would that also mean all editing tools would need to re-sign a photo that was previously signed by the original sensor. How do we distinguish an edit that's misleading vs just changing levels? It's an interesting area for sure, but this inverse approach seems much trickier.

CAI’s Content Credential standard accommodates what you suggest, as far as re-signing/provenance, with a chain kind of approach. It supports embedding “ingredient thumbnails” in an image’s manifest, and/or the image’s manifest can embed or link back to source images that are in turn also signed [2].

It feels like the approach assumes a media environment where a professional wants to provably “show their work,” where authenticity adds value to a skeptical audience.

In that spirit, then, I understand CAI’s intention [0] to be to vest that judgment with the creator, and ultimately the viewer: if my purpose is to prove myself, I’d want to show enough links in the chain that the viewer checking my work can say “oh I see how A relates to B, to C,” and so on. If I don’t want to prove myself, well… then I won’t.

I don’t know Adobe’s implementation well enough to know how often they save a CC manifest, and their beta is vague in just referring to “editing history.” [1] I get the impression that they’re still dialing in the right level of detail to capture by default. Maybe even just “came from Firefly” and “Photoshop wuz here.”

But if I want to prove this Nikon Z9 recorded these pixels at this time and place, or “I am the BBC and yes I published this,” or “only the flying monkey was GenAI, the rest was real” I could conceivably put together a toolchain (independently of Adobe) to prove it in more detail.

[0] https://spec.c2pa.org/specifications/specifications/2.2/spec...

[1] https://opensource.contentauthenticity.org/docs/manifest/und...

[2] https://opensource.contentauthenticity.org/docs/c2patool/doc...

Re: SynthID: A tool to watermark and identify content generated through AI

#69
post #52

It's nice that they explain the "what" (...it is doing) but not the "why". Who is going to use it and for what reasons? Also, if it's essentially a sort of metadata, can't the output generated image be replicated (e.g. screenshot) and thus stripped of any such data?

I've heard of journalists using it to try and figure out whether images sent by sources were generated. In their Nano Banana 2 release blogpost, Google mentioned that SynthID has been used ~20 million times, so there's clearly some interest in identifying AI-generated images.

Re: SynthID: A tool to watermark and identify content generated through AI

#70
post #20

I genuinely feel that in this AI world we need the inverse. That every analogue or digital photo taken by traditional means of photography will need to be signed by a certificate, so anyone can verify its authenticity.

This already exists: https://c2pa.org , https://en.wikipedia.org/wiki/Content_Authenticity_Initiativ... . Support by camera makers is - spotty.

C2PA has lots of problems.

https://www.hackerfactor.com/blog/index.php?%2Farchives%2F10...

Post reply on HN