641–650 of 776 posts
Perhaps it's not actually irony, perhaps it is hypocrisy.
This isn't to say there aren't correct statements in the article, but its framed very strangely.
> "The exact words we choose when writing matter." Then write your own damn text if you care about the exact wording so much
My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…
Local models could probably catch some patterns.
My biggest concern is that checking any text for watermarks requires sending the entire text to Anthropic. And even that is not sufficient, as the text might have been generated with ChatGPT, Gemini, Grok, Mistral, ... So every check requires sending the text to as many AI providers as offer a watermarking detection API, almost all of which have a very dubious track history with obtaining training data through illici…
Would really like to know how their watermarking technique works, and if they can use it to store arbitrary information in the text (I assume they can if only in a limited way). I assume if the text is long enough they could add all kinds of metadata that would then be undetectable as the model that generated the text is the "cryptographic" key that encodes the data. I wonder if you can have another model run over th…
Earlier quoted context omitted.
The paper for it is open. The technique isn't really hiding information in the text itself, but by forcing some of the rolls to follow a specific pattern. LLMs work by estimating the most likely next token, so there's sometimes a list of possible candidates that would all work in the text (e.g. synonyms). At low "temperature", the output is a bit more deterministic and otherwise it's a weighted dice roll of which tok…
What happens if we train models (GPT or human students) using the outputs of a model with text havingbthose watermarks? Is there something preventing the watermark from being learnable?
Earlier quoted context omitted.
I'm sure he honestly appreciates the pushback.
I am sure he is quite happy to have people disagree with him on HN. He usually wears it as a badge of pride. We might even have a follow-up in a couple of days about how these techy weirdos lost the plot.
Earlier quoted context omitted.
I'll be interested to see if people can actually pick out which text is watermarked and which isn't once they introduce it. It won't switch "bananas" to "airplanes". It'll switch "I really enjoy eating bananas" to "I love eating bananas" or similar
That’s my point of the post … I had the feeling that the English text editing skills of Claude went significantly down in August. I was frustrated at first not knowing what they are doing. After I read this post, (seeing they introduced it in August) I think it has to do with the watermarking. Try it on a paragraph … the connections between sentences feel clunky now. I will play with it more and see if that’s really…
Crazy how a smart person like this fails to understand the gumbel softmax technique. It does not affect writing quality at all, provably. The very fact that there is generally no "best next token" with 100% certainty is precisely why the trick works (you cannot watermark a response to "respond with the To be or not to be soliloquy from the first folio Hamlet", for precisely this reason).
> a smart person like this Can you give an example of something smart John Gruber has said or written? Because I can't think of one, but I can think of many dumb ones.
Gruber made an effort to but doesn't fully understand how SynthID works. LLMs select the next word randomly from a set probability distribution, so there is no "best choice" unless you run the LLM with a temperature of 0 which would give out terrible results. Anthropic runs a non-distorting version of SynthID that doesn't change the probabilities of the underlying distribution of tokens. It makes the watermark less likely to work over smaller samples but preserves text quality. I encourage the mathematically inclined to read the paper: