Researchers tested AI watermarks and broke all of them
1–10 of 91 posts
Re: Researchers tested AI watermarks and broke all of them
#2[deleted]
Re: Researchers tested AI watermarks and broke all of them
#3Re: Researchers tested AI watermarks and broke all of them
#4People have been trying to watermark digital media for decades, when there was (still is) a very strong financial incentive to get it working. It never worked. I don’t think it ever will work.
Re: Researchers tested AI watermarks and broke all of them
#5People have been trying to watermark digital media for decades, when there was (still is) a very strong financial incentive to get it working. It never worked. I don’t think it ever will work.
“Information wants to be free.”
Re: Researchers tested AI watermarks and broke all of them
#6It’s like captcha, highly annoying to users and authors, but if you don’t want to pay it works against low spend bots
Re: Researchers tested AI watermarks and broke all of them
#7People have been trying to watermark digital media for decades, when there was (still is) a very strong financial incentive to get it working. It never worked. I don’t think it ever will work.
You can still declare success if you lower the bar to "we can catch leaks/pirates and in particular we can know which internal folks should no longer be trusted. ... as long as they don't attempt to circumvent the fingerprint"
Re: Researchers tested AI watermarks and broke all of them
#8For written text, the problem may be even harder. Identifying the human author of text is a field called "stylometry" but this result shows that some simple transformations reduce the success to random chance [1].
Similarly, I suspect watermarking LLM output is probably unworkable. The output of a smart model could be de-watermarked by fine tuning a dumb open source model on the initial output, and then regenerating the original output token by token, selecting alternate words whenever multiple completions have close probabilities and semantically equivalent. It would be a bit tedious to perfectly dial in, but I suspect it could be done.
And then ultimately, short text selections can have a lot of meaning with very little entropy to uniquely tag (e.g., covfefe).
[1] https://dl.acm.org/doi/abs/10.1145/2382448.2382450
Curious if Scott Aaronson solved this challenge...
Re: Researchers tested AI watermarks and broke all of them
#9It seems it would be much easier to watermark non-ai images instead. Aka crypto signature.
That will be much harder to evade, but also pretty hard to implement.
I guess we will end up in the middle ground, where any non-signed image could be ai generate, but for most day to day use it’s ok.
If you want something to be deemed legit (gov press release, newspaper photo, etc) then just sign it. Very similar to what we do for web traffic (https)
Re: Researchers tested AI watermarks and broke all of them
#10We need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that can be used by downstream systems to _better_ detect fraud, though not eliminate it.