What a stupid take. Generating text with an LLM is already a ‘perversion’ of writing. Tweaking the last random-choice step doesn’t meaningfully change that at all.
Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
541–550 of 776 posts
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#542Watermarking seems feasible for creative tasks (that admit many valid results).
But if I craft a prompt that doesn't leave space for creativity, how can they include a watermark? e.g. "Rewrite the following text, replacing 'foo' with 'bar'."
I'm curious to see where they draw the line, and whether the watermarking really affects the (perceived) quality of results.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#543Earlier quoted context omitted.
I can't remember any radio determining words or adjusting grammar of the person speaking through it. Neither can I recall there has ever been a moveable type press, laser printer, or inkjet which bastardised the words of authors. These were mediums _through_ which communication happened. Language models, large or small, are not any such medium.
> Neither can I recall there has ever been a moveable type press, Ah my friend, you are about to fall down a rabbit hole into standardisation of spelling, and the sometimes deadly debates about how to translate latin into the vernacular. English, as she is written, is a great example. for the spoken word, BBC/received pronunciation is another. I speak the way I do _because_ of BBC radio. The reason I have the accent…
A radio device at BBC did all this and not humans?
> again, your argument is against LLMs and globalisation of culture, not finger printing.
ABSOLUTELY NOT. Do not ever put words in my mouth. My argument is that radio is a medium through which humans communicate. An LLM is not.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#544Earlier quoted context omitted.
Doesn’t this mean Anthropic can accuse anyone of using their AI to write for them?
Seems like exactly the sort of problem the threat of defamation lawsuits are meant to solve.
Evidence is a massive problem here. As well as the extremely high threshold for US defamation; political candidates routinely tell the most absurd lies about each other.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#545> "The exact words we choose when writing matter." Then write your own damn text if you care about the exact wording so much
The exact words matter to people who people who don’t use it to write for them. A lot of people use Claude as a friend/therapist/romantic partner/etc. That’s who I imagine would be most affected
How would you ever know if you've been affected? How, indeed, would you know at all whether the word you get next is different from some different word next?
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#546Gruber is a smart, thoughtful man. This is a bizarre take from him.
It exhibits an undeveloped understanding of LLMs, and a righteous view that generated prose should assimilate... which should be offensive to organic intelligence.
Issues with the proprietary nature of Anthropic's watermarking aside, we will look back on this as a 'thank god' moment in the history of LLMs.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#547Earlier quoted context omitted.
I can't remember any radio determining words or adjusting grammar of the person speaking through it. Neither can I recall there has ever been a moveable type press, laser printer, or inkjet which bastardised the words of authors. These were mediums _through_ which communication happened. Language models, large or small, are not any such medium.
> Neither can I recall there has ever been a moveable type press, Ah my friend, you are about to fall down a rabbit hole into standardisation of spelling, and the sometimes deadly debates about how to translate latin into the vernacular. English, as she is written, is a great example. for the spoken word, BBC/received pronunciation is another. I speak the way I do _because_ of BBC radio. The reason I have the accent…
There are two different things being talked about here, one of which is "what effect does widespread LLM use have on a culture", and one of which is "what is the effect on a specific text of running it through an LLM".
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#548Earlier quoted context omitted.
But if the user's prompt is in the context, you don't know exactly what the RNG chooses between. I don't know what trick they use to get past that, but it seems impossible to get by it in the general case (i.e. if the prompt can be anything) and you'll probably quickly compromise quality if you try.
A reasonable guess about the algorithm is 'A Watermark for Large Language Models' ( https://arxiv.org/abs/2301.10226 ). The idea is that each generated token (or bigram) seeds a strong PRNG that splits the vocabulary into a 'green' and 'red' set. The sampler then tries to select a 'green' next-token for generation. After-the-fact checking only needs the vocabulary splitter, which is independent of the LLM. Over a suf…
Anthropic actually tells you the approach they use, and it's not that. From their Claude Text Watermark page[0]: "Claude’s text watermark is a version of the SynthID-Text approach published by Google DeepMind in a Nature paper in 2024."
The Nature paper is "Scalable watermarking for identifying large language model outputs"[1]. This method does not separate out tokens into separate classes, but merely uses a seed for the PRNG that selects which among the most likely tokens generated by the LLM will actually be output. This has the advantage that there's no green and red token sets, so no token is systematically favored or disfavored. If a particular token is overwhelmingly predicted to be the most likely candidate, it will almost certainly be selected, so the watermark doesn't affect that. Even if there are several choices of output token at a point that have similar probability of selection, the watermark doesn't systematically bias in favor of one token or the other.
This is actually a quite elegant method of watermarking that, contrary to people's fears, won't adversely affect the model output. The main concern I have with it is that it appears that you can't actually test the watermark locally, without uploading it to Anthropic. I'm not sure why that's the case, since there's no particular reason the watermarking key has to be private, except if you want to prevent others from generating text with their own LLMs that is watermarked to look like it's generated by Anthropic - but everybody wants their text to not have the watermark.
[0]: https://www.anthropic.com/news/claude-text-watermark#:~:text...
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#549Watermarking seems feasible for creative tasks (that admit many valid results). But if I craft a prompt that doesn't leave space for creativity, how can they include a watermark? e.g. "Rewrite the following text, replacing 'foo' with 'bar'." I'm curious to see where they draw the line, and whether the watermarking really affects the (perceived) quality of results.
Although that feels like an overly simplistic example to the point of not being not being a helpful example, I agree that the amount of tokens to encode the watermark is so low in text, that it wouldn't take much to erase or distort it.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#550Watermarking seems feasible for creative tasks (that admit many valid results). But if I craft a prompt that doesn't leave space for creativity, how can they include a watermark? e.g. "Rewrite the following text, replacing 'foo' with 'bar'." I'm curious to see where they draw the line, and whether the watermarking really affects the (perceived) quality of results.
They can't include a watermark in that case, that's not the relevant use case.