Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta's Un-Stable Signature

hackerfactor.com

Post reply on HN
Print view View original

21–28 of 28 posts

Re: Meta's Un-Stable Signature

#21

Post by nocoolnametom »

UltraSane wrote: post #19
itake wrote: post #7

A watermark is not just “transparency.” It can reveal what tool someone used, how they work, or that an image came from a stigmatized platform. In sensitive contexts—politics, sexuality, medical issues, protest material, or private expression—that can become surveillance. I am working on Saigon Watermarks: https://apps.apple.com/us/app/saigon-watermark/id6777061197 for detecting and removing provenence markers in AI.…

When generative AI can create such good fake images a valid c2pa linked to the source camera will become mandatory for an image to be considered authentic.

This is what REALLY pains me about this discussion: I am 100% about personal digital freedom, but I am also 100% opposed to promoting political violence and promoting theft and grift using generative AI. If C2PA is going to work towards one goal by being diametrically opposed to the other then it _cannot_ be a useful tool and we need an _actual_ solution. I was extremely excited by C2PA until today and now am only disappointed that there isn't already some better solution.

Edit: Thinking through this a bit more, I think the goal of _authenticating_ a photo using C2PA is still useful. If the goal is to remove them to get a "naked" image, that's fine, such an image is then inherently no more or less trustworthy than any other image. If the goal is to figure out how to reproduce a valid provenance chain on top of an altered image then I have problems with that.

Re: Meta's Un-Stable Signature

#22

Post by kamranjon »

How common is it for peer reviewed papers like this to be so far off their claimed findings?

“According to Google's peer-reviewed and published paper, they claim to have a true positive rate (TPR) above 99.97% -- meaning that they will miss their own watermarks less than 1 in 10,000 times. However, my own empirical testing found that is it much closer to 1 in 20.”

Re: Meta's Un-Stable Signature

#23

Post by a34729t »

kamranjon wrote: post #22

How common is it for peer reviewed papers like this to be so far off their claimed findings? “According to Google's peer-reviewed and published paper, they claim to have a true positive rate (TPR) above 99.97% -- meaning that they will miss their own watermarks less than 1 in 10,000 times. However, my own empirical testing found that is it much closer to 1 in 20.”

If there were bounties for invalidating peer reviewed research, I suspect this would be a lot leas common.

Re: Meta's Un-Stable Signature

#24

Post by f33d5173 »

The point is to embed a particular signature, which was generated randomly, in the image. The distance function discussed is the same as the popcount of the xor. It's well know that the xor of random data with correlated data is statistically random. Hence, however well correlated the signatures of unwatermarked images may be with each other, they would show no correlation with the signature of a watermarked image. That is, unless the watermark by extremely bad luck happened to be near one of these clusters the author discovered. This does represent a genuine flaw, but an extremely minor one, and one that can be easily mitigated with no changes to the underlying algorithm,

Re: Meta's Un-Stable Signature

#25

Post by mort96 »

hparadiz wrote: post #17
itake wrote: post #7

A watermark is not just “transparency.” It can reveal what tool someone used, how they work, or that an image came from a stigmatized platform. In sensitive contexts—politics, sexuality, medical issues, protest material, or private expression—that can become surveillance. I am working on Saigon Watermarks: https://apps.apple.com/us/app/saigon-watermark/id6777061197 for detecting and removing provenence markers in AI.…

Another reason to drop both iOS and Android.

In favour of what, might I ask?

Keep in mind that the replacement must be a first class citizen in the society I live in. That means being able to easily transfer money using the app everyone uses where I live. It means public transport ticket apps. It means banking apps. It means iMessage, Facebook Messenger, Snapchat and Signal.

Re: Meta's Un-Stable Signature

#27

Post by hackerfactor1 »

richardfey wrote: post #6

This is a great statistical analysis and it was a pleasure to read, but I wasn't expecting the claims to be so poorly supported. There's also a reply from one of the Meta authors there, worth checking out.

Which claims do you think are poorly supported? I'm the author; I tried to include everything other people need to repeat the same experiments. I've even had two people write in directly to me, stating that they have been able to replicate my findings.

Or are you referring to the claims from Meta, Google, and Adobe -- which failed to hold up under independent evaluation.

You are also correct that one of the meta authors wrote in a comment. However, he demonstrated a clear lack of understanding regarding what makes the bits "independent" or how to resolve the independence problem.

Re: Meta's Un-Stable Signature

#28

Post by richardfey »

hackerfactor1 wrote: post #27
richardfey wrote: post #6

This is a great statistical analysis and it was a pleasure to read, but I wasn't expecting the claims to be so poorly supported. There's also a reply from one of the Meta authors there, worth checking out.

Which claims do you think are poorly supported? I'm the author; I tried to include everything other people need to repeat the same experiments. I've even had two people write in directly to me, stating that they have been able to replicate my findings. Or are you referring to the claims from Meta, Google, and Adobe -- which failed to hold up under independent evaluation. You are also correct that one of the meta auth…

> Or are you referring to the claims from Meta, Google, and Adobe -- which failed to hold up under independent evaluation.

This.

> However, he demonstrated a clear lack of understanding regarding what makes the bits "independent" or how to resolve the independence problem.

Yes, I didn’t want to call it out explicitly, but this is exactly the kind of thing that would have made my undergraduate statistics professor lose patience.

Post reply on HN