> But unless you know the prompt, you don't fully know which choice the model faces. Surely a lot of coding space is wasted compensating for that uncertainty. When you only need to encode one bit, the signal to noise ratio can be very low. If I try to write my own human words under the policy of "try a little bit to avoid the letter 'e' in every fifth word," then a sufficiently long text would still be 'watermarked'…
Whereas if you fully avoid the letter e, everyone will know you are George Perec
And if you lowercase everything, you're e e cummings.
So the watermark can be removed by rearranging words and choice of words. This seems trivial to bypass with a local model. If I understand this correctly.
my understanding is that it's a probability so it can never really be removed? only made smaller.
1. As watermarked text is added to the training data, watermark-related tokens will be associated more with AI outputs and thus lower quality outputs which will hasten model collapse. Especially because every provider has its own secret key and they are all training on eachother's outputs anyway. I guess they can at scale filter the watermarked documents (by necessarily allowing eachother to at scale checked for wate…
There are no watermark related tokens, there is no watermarked set - read the paper, it's public and not that complicated. It doesn't change the distribution of completions, and won't change the situation where one output token has the majority of probability mass. You're missing the fundamentals here!
I am going off the explanation in the declaude page (and related papers). But I see now anthropic mentions Aaronson's distortion-free watermarking.
Random watermarking functions colour the tokens based on (small) contexts and a secret key. Given watermarking functions are randomly chosen every single time they are used (so essentially not deterministically seeded by the context and secret key), then indeed the completion distributions are unchanged. However, they are deterministically chosen, a given string of text will always have the same corresponding watermarking functions. Tokens scored high by the function see an increased probability of being the chosen completion, those scored low see a reduced probability. I dont see it is different from merely talking about it as green/red and the points hold?
> After all, i am allowed to learn off anything i can legally read (and perhaps even illegally read). Are you a tool? Because humans gets rights, tools don't. Arguing that untrained or partially trained models should have have rights is a different argument to arguing that a trained model should get the same rights as a human.
What if I'm reading it for work? What am I but a tool for the corporatioN?
Are you claiming that you have no human rights while you are at work?
Text watermarking is another EU rule made without real world input. The Union is stuck on major economic crises (electricity prices for instance) because nobody can agree on anything. However, the bureaucracy forces tech into a privacy nightmare. Brussels cannot bring together its own members but it loves pretending it can govern the internet.
> (electricity prices for instance) because nobody can agree on anything. I would say that's more like because the US has arranged for Europe's fossil fuel energy sources to be disrupted or cut off: * Libya - NATO made a pig's breakfast of that, it's a failed state now. * Iran - transitive sanctions, because why not prevent non-US states from trading with each other. * Russia (& Kazahkhstan) - The US (with or without…
Spot on. Also, in 2003, the war on Iraq, still occupied. And the proxy war on Syria (stifling an unwelcome pipeline project). European “leaders” pretend to not comprehend how they're being screwed. Stockholm syndrome. Populations don't understand, propaganda (“free press”) working correctly.
Would really like to know how their watermarking technique works, and if they can use it to store arbitrary information in the text (I assume they can if only in a limited way). I assume if the text is long enough they could add all kinds of metadata that would then be undetectable as the model that generated the text is the "cryptographic" key that encodes the data. I wonder if you can have another model run over th…
The paper for it is open. The technique isn't really hiding information in the text itself, but by forcing some of the rolls to follow a specific pattern. LLMs work by estimating the most likely next token, so there's sometimes a list of possible candidates that would all work in the text (e.g. synonyms). At low "temperature", the output is a bit more deterministic and otherwise it's a weighted dice roll of which tok…
What happens if we train models (GPT or human students) using the outputs of a model with text havingbthose watermarks? Is there something preventing the watermark from being learnable?
Essentially yes. Flac takes a raw .wav and effectively zips it up to shave off a certain amount of space. (there are nuances, I think the compression scheme is designed for streaming.) mp3 is perceptual, so throws away the stuff that humans can't hear. This yields a much smaller file. However its all a sliding scale like PNG vs jpeg. a .jpg with a quality setting of 85 will be almost identical to a .png in visual qua…
Interestingly, in a test c't magazine did a long time ago, the test audience preferred mp3 256kbit over the original.
Could be c't magazine accidentally played the mp3 version louder. Human's have a known preference for louder music, and will tend to prefer louder samples over quieter samples. Rumor in the industry is that this was a trick MS used to try to push the WMA format, that they encoded some WMA samples used in some publicized tests at +3dB above the source sample.
As a university professor, back in the early days of this when students were just copy / pasting output directly I was really hoping the model builders would implement steganographic fuzzy hashes.
Not that it's the ideal solution. But it would be really neat.
All those methods are applied with the specific goal of improving output quality and are applied to the extent that they do this. Watermarking has no such goal, and is not implemented for any such reason. In fact it's much more like applying another layer of random noise over the token selection process, because the sequence that generated the green token list comes from a seeded PRNG. >My second point is that the tr…
> All those methods are applied with the specific goal of improving output quality and are applied to the extent that they do this Yes, that's the goal that was used, but they are quite simplistic and crude methods, not some specifically designed function, with carefully fine tuned parameters or something. So, if a basic function like top_k can improve model utility, it's not impossible to imagine that watermarking c…
But as I explained, the watermark is functionally random PRNG noise overlaid on the token probabilities. It’s not something that can be compensated for because it’s not predictable if you don’t have the seed and PRNG function.