gruber is really out of his element with ai commentary, I fully support the general skepticism but he's seemingly arguing against something he doesn't quite grasp
Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
91–100 of 776 posts
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#92Reminder: your favorite distilled model does not treat you, the customer, as an adversary and mess with your output.... May the free market win.
[deleted]
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#93Earlier quoted context omitted.
I think this is a key reason why humans write better prose than LLMs - we can try to choose the best word every time, and go back and restructure sentences and paragraphs if we want. On the other hand, LLMs are forced into picking some likely-ish word, and then have to build the rest of their response to retcon that choice into making sense. Even good human writers would probably struggle with this constraint. It wou…
This is the classic misunderstanding that LLMs only pick the next token at a time. Really, they are coalescing the probabilities of a range of tokens at a time. There is no “oops, I wrote ‘th’ but I should have written ‘tw’ so I guess I’m stuck writing three instead of tween”.
everyone in this thread is saying something kind of imprecise and reductive and varies between models and even modifications among the models
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#94> I want any LLM I use to choose the very best, most precise words at every single decision point. Then bad news: LLMs already use randomness in a fundamental way. Each time they go to generate a token, they first generate a probability distribution of possible tokens. Then they pick one randomly according to this distribution. The technique described can be thought of as making the random number generator pseudo ran…
Yeah this is my main issue with the argument.
He acknowledges in the article that LLMs are already non-deterministic, but he doesn’t seem to actually understand that.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#95Earlier quoted context omitted.
That's inaccurate in two ways: (1) The behavior that is approximately what you describe is not "fundamental" (though it may not be something you can disable on some hosted providers), it is an option that is not fundamental (and with runtimes where you have full control can be either disabled or tuned in a large number of manners), and (2) The actual behavior that is approximately what you describe already usually in…
(1) LLMs collapse and start outputting garbage after a number of tokens if you do not sample and just pick the "best token" each time. This is a consequence of how they are trained.
Citation needed
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#96Earlier quoted context omitted.
Perhaps LLM outputs are uncopyrightable, but derivative works of copyrighted works are not automatically in the public domain.
That's an intriguing twist, isn't it? It could lead to a tug-of-war. Working backwards: if it is possible to confirm 100% confidence that a chunk of text is LLM output, then it is "PD until proven otherwise". How can a human reliably assert human authorship of their source text? When all watermark tests fail? Is that proof of humanity now? If a human proves human authorship, and LLM watermarking tests positive, then…
The watermark, if I'm understanding correctly, only proves that something has been touched at some point by a particular model, not that it was entirely generated by a particular model.
As for it being "uncopyrightable" if it were the output of an LLM, I think computer people are making a very aspie interpretation of a single decision. I think it's more that the LLM (and thereby its owners) cannot itself hold a copyright on its output, that output has to be touched by a person before it is copyrightable. A particular view from the top of a mountain can't be copyrighted, for example, but a photograph of that view can be.
I'm not sure it at all precludes a "robosigning"* sort of situation, where machines generate output, hired temps sign and claim that output, and immediately sign it over to the people who hired them (as a work-for-hire.) Copyright is stupid, artificial law, not logical.
-----
* https://www.mortgageauditsonline.com/what-are-robo-signers/
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#97> One of my fundamental problem with this is that no two synonyms carry the exact same meaning. “He leaped at the chance” and “He jumped at the opportunity” are very similar sentences expressing the same general sentiment, but they are not the same. The exact words we choose when writing matter.
Then why are you using an LLM to write? They're not capable of understanding such nuance. They do pick randomly between two synonymous phrases, they do not use some super smart algorithm to pick the one that sounds the best.
This excuse doesn't hold any water at all - Occam's razor says the author is just super annoyed that his AI writing will be identifiable as AI writing.
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#98Translation: No one can ever again use Claude for proofreading their own prose unless they’re willing to risk that the whole thing might be flagged as having been generated by Claude. I think that was intended, yes.
I'm not sure the quoted statement is true. Proofreading like "point to problems in the text", if you fix the problems yourself and don't copy-paste the solutions given to you, should still be safe, shouldn't it? So, human-written text should not be falsely flagged if you use LLM for proofreading. And if you copy-paste the answers from LLM, I think it's only fair the end result gets flagged. You're not writing it your…
> And if you copy-paste the answers from LLF, I think it's only fair the end result gets flagged. You're not writing it yourself.
I wonder if it would even get flagged in that case, because wouldn't the probability distribution of a token when the LLM is suggesting an edit to your writing be different than the distribution of that token once it is in the context of the text it's editing?
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#99Earlier quoted context omitted.
That's missing the point. It's the distribution that's the "best", not the tokens. Then Anthropic comes in and makes the distribution something other than the best. The only saving grace is that Anthropic says it's not that bad. Even so, I don't think it will stop here. Once this is in place, the next step is to put more and more identification into the AI generated content; might as well pack it in, it's not that ba…
The watermark doesn’t change the distribution, only per-token selection. I think not understanding that is the source of most people’s FUD.
This comment disagrees with you: https://news.ycombinator.com/item?id=49324387
Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing
#100> I want any LLM I use to choose the very best, most precise words at every single decision point. Then bad news: LLMs already use randomness in a fundamental way. Each time they go to generate a token, they first generate a probability distribution of possible tokens. Then they pick one randomly according to this distribution. The technique described can be thought of as making the random number generator pseudo ran…
I think this is a key reason why humans write better prose than LLMs - we can try to choose the best word every time, and go back and restructure sentences and paragraphs if we want. On the other hand, LLMs are forced into picking some likely-ish word, and then have to build the rest of their response to retcon that choice into making sense. Even good human writers would probably struggle with this constraint. It wou…
It's a bit like trying to finish a sentence when you're really stoned... you vaguely remember the preceding couple of words you've said but don't really know how you got there and now you're wandering in the forest trying to stumble on coherency.
Well, I suppose it's nearly the opposite of that experience, upon further review. But for some reason, that's where my head jumped.