Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reverse engineering Gemini's SynthID detection

github.com

Post reply on HN
Print view View original

31–40 of 66 posts

Re: Reverse engineering Gemini's SynthID detection

#31

Post by Diggsey »

levocardia wrote: post #30
raincole wrote: post #21

Earlier quoted context omitted.

There is no solution . I don't know why people discuss this subject as if there is a technical solution. As if there are fairies or souls hidden in the pixels that help us tell what is AI generated and what is not.

Sure there is a solution, you are just looking at it the wrong way. Make non-AI images provably unaltered with signed keys from the device (e.g. the camera) that took it.

Which works for about 5 minutes until someone leaks a manufacturer's private key or extracts it from a device...

Re: Reverse engineering Gemini's SynthID detection

#32

Post by IncreasePosts »

levocardia wrote: post #30
raincole wrote: post #21

Earlier quoted context omitted.

There is no solution . I don't know why people discuss this subject as if there is a technical solution. As if there are fairies or souls hidden in the pixels that help us tell what is AI generated and what is not.

Sure there is a solution, you are just looking at it the wrong way. Make non-AI images provably unaltered with signed keys from the device (e.g. the camera) that took it.

How many minutes do you think it would take before someone figured out how to crack that?

Re: Reverse engineering Gemini's SynthID detection

#33

Post by coppsilgold »

Inserting an undetectable 1-bit watermark into a multi megapixel image is not particularly difficult.

If you assume competence from Google, they probably have two different watermarks. A sloppy one they offer an online oracle for and one they keep in reserve for themselves (and law enforcement requests).

Also given that it's Google we are dealing with here, they probably save every single image generated (or at least its neural hash) and tie it to your account in their database.

Re: Reverse engineering Gemini's SynthID detection

#34

Post by subscribed »

lokar wrote: post #14
khernandezrt wrote: post #10

Ok i get that eventually someone was gonna do this but why would we want to purposely remove one of the only ways of detecting if an image is ai generated or not...?

It was always going to be available to some people, but not everyone would know or believe that. Now they will.

More likely than not it would be used to deanonymise the author.

So it's a "no" by default.

Re: Reverse engineering Gemini's SynthID detection

#35

Post by subscribed »

IncreasePosts wrote: post #32
levocardia wrote: post #30

Earlier quoted context omitted.

Sure there is a solution, you are just looking at it the wrong way. Make non-AI images provably unaltered with signed keys from the device (e.g. the camera) that took it.

How many minutes do you think it would take before someone figured out how to crack that?

On Pixels and iPhones it would be impossible since they have actually secure hardware that could both hold the keys and sign/verify the image.

Re: Reverse engineering Gemini's SynthID detection

#36

Post by ddtaylor »

Tiberium wrote: post #29

Seems like a very low-quality AI-assisted research repo, and it doesn't even properly test against Google's own SynthID detector. It's not hard at all (with some LLM assistance, for example) to reverse-engineer network requests to be able to do SynthID detection without a browser instance or Gemini access, and then you'd have a ground truth.

I read a lot of comments on HN that say something is not hard, yet don't provide a POC of their own or link to research they have knowledge of.

I also read a lot of comments on HN that start by attacking the source of the information, such as saying it was AI assisted, instead of the actual merits of the work.

The HN community is becoming curmudgeonly and using AI tooling as the justification.

Re: Reverse engineering Gemini's SynthID detection

#37

Post by dummydummy1234 »

DonsDiscountGas wrote: post #25
raincole wrote: post #21

Earlier quoted context omitted.

There is no solution . I don't know why people discuss this subject as if there is a technical solution. As if there are fairies or souls hidden in the pixels that help us tell what is AI generated and what is not.

If you want to make an AI generated image but don't want other people to know that it's AI, the most obvious solution is to not use Gemini. Synth ID is watermarking. It's only ever going to be useful to good actors, who want an AI generated image and aren't trying to hide the fact that it's AI generated.

Never underestimate that people are lazy.

Re: Reverse engineering Gemini's SynthID detection

#38

Post by IncreasePosts »

subscribed wrote: post #35
IncreasePosts wrote: post #32

Earlier quoted context omitted.

How many minutes do you think it would take before someone figured out how to crack that?

On Pixels and iPhones it would be impossible since they have actually secure hardware that could both hold the keys and sign/verify the image.

The camera module sits outside the secure area, meaning it would need to send data in to be signed. How does the phone know that it's getting legitimate data from the camera module, or data someone else is just piping in? Also, you could probably get a fairly high quality image by just taking a photo of something AI generated in the right lighting conditions.

Re: Reverse engineering Gemini's SynthID detection

#39

Post by raincole »

levocardia wrote: post #30
raincole wrote: post #21

Earlier quoted context omitted.

There is no solution . I don't know why people discuss this subject as if there is a technical solution. As if there are fairies or souls hidden in the pixels that help us tell what is AI generated and what is not.

Sure there is a solution, you are just looking at it the wrong way. Make non-AI images provably unaltered with signed keys from the device (e.g. the camera) that took it.

If the premise is that everyone would just agree on the same protocol, I have an even more unbreakable solution: every image has to be upload to a blockchain the moment it is (claimed to be) created. Otherwise it's AI.

If only everyone just agrees with me.

Re: Reverse engineering Gemini's SynthID detection

#40

Post by jfim »

levocardia wrote: post #30
raincole wrote: post #21

Earlier quoted context omitted.

There is no solution . I don't know why people discuss this subject as if there is a technical solution. As if there are fairies or souls hidden in the pixels that help us tell what is AI generated and what is not.

Sure there is a solution, you are just looking at it the wrong way. Make non-AI images provably unaltered with signed keys from the device (e.g. the camera) that took it.

That's pretty much impossible though.

One workflow that some artists use is that they draw with ink on paper, scan, and then digitally color. Nothing prevents someone from generating line art using generative AI, printing it, scanning it, and coloring it.

And what if someone just copy pastes something into Photoshop or imports layers? That's what you'd do for composites that mix multiple images together. Can one copy paste screenshots into a multi layer composition or is that verboten and taints the final image?

And what about multi program workflows? Let's say I import a photo, denoise it in DxO, retouch in affinity photo, resize programmatically using image magick, and use pngcrush to optimize it, what metadata is left at the end?

Post reply on HN