Earlier quoted context omitted.
Are there any diffusion-based or otherwise non-transformer-based models in mainstream use?
If you include non-language models, yes.
We are clearly talking about language models
Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
731–740 of 776 posts
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#732Earlier quoted context omitted.
Seems like exactly the sort of problem the threat of defamation lawsuits are meant to solve.
Evidence is a massive problem here. As well as the extremely high threshold for US defamation; political candidates routinely tell the most absurd lies about each other.
In places like Germany it's a crime to say something that makes a politician look bad, even if it's true.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#733Earlier quoted context omitted.
It's exactly the same problem as with humans, though. It's part of why we sign NDAs, and why their duration is measured in years (and that's not even targeting the human retention - just duration after which information ages enough that its disclosure is not likely to negatively impact anyone who cares).
Weirdly I think humans should have more rights than machines
Yes, but in a big way, denying capabilities to LLMs is denying the capability for people to exercise the rights they have. LLMs are user agent tools.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#734Crazy how a smart person like this fails to understand the gumbel softmax technique. It does not affect writing quality at all, provably. The very fact that there is generally no "best next token" with 100% certainty is precisely why the trick works (you cannot watermark a response to "respond with the To be or not to be soliloquy from the first folio Hamlet", for precisely this reason).
It seems to me like he started out mad and looked to justify it. I'm skeptical that anybody generating LLM text is really all that concerned about optimal word choice. Or even particularly good prose. But let's pretend that person exists. If that person tried, say, an open model and that same model with watermarking applied, I'd be eager to hear their thoughts on the prose quality. Especially if they built an experim…
I care about optimal word choice when generating LLM texts. Because my use case is almost exclusively reading the generated text not posting it. I use LLMs to summarize, translate and review other texts. When using LLMs in that way, as a research tool watermarking is a pointless and should not get in the way of "optimal" results.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#735Earlier quoted context omitted.
Thank you for stating clearly situation. I fully agree with your assessment. For almost 10 years now I have been saying that we need to virtually watermark reality. By "virtual" I mean store the metadata about the digital capture on a public blockchain. Then my devices could have a built-in "fake vs real" detector. Artists, photographers, journalist, etc. are going to want and need this.
Some sort of signatures to verify legitimate unmodified photographs (taken with a relatively secure device, e.g an iPhone) seems like a reasonable idea But even then, people will be able to point that camera at a manipulated/generated image (either printed or on a screen). Maybe that one could be solved if the photo included some depth information?
I doubt that NYT, WaPo, or any other mainstream journalist would do that.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#736Earlier quoted context omitted.
It doesn't, though. From the outside, without access to the parameters, you can't distinguish the watermarking system from a random number generator. We already use an RNG at inference precisely because it leads to higher-quality output . Changing what function is generating our random numbers changes the sequence, not the randomness from the point of view of a user. Fundamentally the article is railing against --tem…
>It doesn't, though. From the outside, without access to the parameters, you can't distinguish the watermarking system from a random number generator. You can't as a user tell by how much the quality of the output was degraded. True. >We already use an RNG at inference precisely because it leads to higher-quality output. Changing what function is generating our random numbers changes the sequence, not the randomness…
> I said that the quality of the system is in the quality of the probabilities. That quality is being degraded.
How is it being degraded exactly? The probability that it picks each option will still be the same, just deterministic based on a seed generated from the text.
LLMs already use PRNGs. This is just changing the source of the seed. And a different seed does not change the "quality" of the random numbers. Even if you are worried that it somehow might, they can just use a cryptographic PRNG, then it is literally guaranteed that the source of the seed will not affect the output in any noticable way.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#737> “He leaped at the chance” and “He jumped at the opportunity” are very similar sentences expressing the same general sentiment, but they are not the same. The exact words we choose when writing matter. I want any LLM I use to choose the very best, most precise words at every single decision point. Neither of these is "better" or "more precise"; in fact, LLMs will generally choose randomly between these candidates ba…
I’m more concerned that this will negatively impact code generation. An additional constraint completely unrelated to code quality is unacceptable as far as I’m concerned. I was an Anthropic user but now I’m looking at OpenAI or even better, open models.
Unless you're already manually setting temperature to zero on the models you work with, the deviation from the "optimal" path imposed by the watermark should be no greater than what you already get from the randomization.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#738Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#739I'm doing a fiction project using AI to write the framework, then fixing it. AI writes terrible fiction. Every single sentence in the project is approved and/or changed by me. And I like em dashes for this project. I've been using em dashes since TeX came out.
The question is: does this make my project "AI" or does it make it mine?
People use professional editors all the time. But they agree to give up any copyright in their edits. If I approve a change made by an editor, it becomes mine. But there is this idea that approving an AI sentence still make it AI.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#740Earlier quoted context omitted.
It seems to me like he started out mad and looked to justify it. I'm skeptical that anybody generating LLM text is really all that concerned about optimal word choice. Or even particularly good prose. But let's pretend that person exists. If that person tried, say, an open model and that same model with watermarking applied, I'd be eager to hear their thoughts on the prose quality. Especially if they built an experim…
I care about optimal word choice when generating LLM texts. Because my use case is almost exclusively reading the generated text not posting it. I use LLMs to summarize, translate and review other texts. When using LLMs in that way, as a research tool watermarking is a pointless and should not get in the way of "optimal" results.
Again, given the limits of LLMs (stochastic, rapidly changing, everything's a hallucination, widely known prose issues) I am skeptical that you really care that much about optimal prose. I could believe it's one of the things that you care about, but at a pretty low priority level.
Taking you at your word, though, I'd be interested to see what you think of the watermarking technology in a blind A/B test.