681–690 of 776 posts
Earlier quoted context omitted.
Yeah, wait for LLM "scrambles" that put every paragraph and then the whole text through multiple re-write/edit style cycles.
How would that change anything? The proposed watermark is applied while the output tokens are being chosen, taking that text and running it through an LLM again would just repeat the process.
I don’t see how this makes it worse.
Earlier quoted context omitted.
> All those methods are applied with the specific goal of improving output quality and are applied to the extent that they do this Yes, that's the goal that was used, but they are quite simplistic and crude methods, not some specifically designed function, with carefully fine tuned parameters or something. So, if a basic function like top_k can improve model utility, it's not impossible to imagine that watermarking c…
But as I explained, the watermark is functionally random PRNG noise overlaid on the token probabilities. It’s not something that can be compensated for because it’s not predictable if you don’t have the seed and PRNG function.
Earlier quoted context omitted.
Yes, I decided to stop reading his blog relatively recently after some extremely hot takes on EU policy. I don't feel his thoughts on the matter are particularly well-thought-out, and I feel like he's just stanning for Apple from his priors rather than from any grounding in reality. I dunno, I guess that's what you should expect from Gruber but these EU-bashing articles lowered the enjoyment I got from his blog under…
Freaking out about basic EU privacy and safety measures is always one of the biggest red flags that someone is becoming a politically motivated hack, especially when that someone lives in the US where there is an active fascist movement tearing apart the government.
Earlier quoted context omitted.
You’re not getting it. The probabilities do not change at all. The only change is given some probabilities there is a deterministic method for determining which symbol was sampled from that distribution. The distribution or sampling process itself is not modified.
Based on the SynthID-Text paper https://www.nature.com/articles/s41586-024-08025-4 I agree that the LLM's learned distribution isn't modified, but I don't think it's correct to say that the sampling process is not modified. Also I just read the paper today so I could be misinterpreting things. As described in the paper, you're right that it doesn't affect the main sampling technique, but what they do is they sample t…
> “By definition it must make text worse … because the nature of the watermarking algorithm requires it to sometimes increase the probability of selecting a worse word choice and decrease the probability of selecting the model’s best choice.” Gruber made an effort to but doesn't fully understand how SynthID works. LLMs select the next word randomly from a set probability distribution, so there is no "best choice" unl…
I came here to quote the same sentence. Here's another way to look at it: Suppose there actually is a best word choice. The LLM doesn't know what it is but makes a guess. Maybe it's the best one, maybe it isn't. The probability that SynthID changes the best choice to a worse one is equal to the probability that it changes a worse choice to the best one.
> “By definition it must make text worse … because the nature of the watermarking algorithm requires it to sometimes increase the probability of selecting a worse word choice and decrease the probability of selecting the model’s best choice.” Gruber made an effort to but doesn't fully understand how SynthID works. LLMs select the next word randomly from a set probability distribution, so there is no "best choice" unl…
I'm not understanding how the word with the highest probability isn't the "best choice"?
Earlier quoted context omitted.
I am going off the explanation in the declaude page (and related papers). But I see now anthropic mentions Aaronson's distortion-free watermarking. Random watermarking functions colour the tokens based on (small) contexts and a secret key. Given watermarking functions are randomly chosen every single time they are used (so essentially not deterministically seeded by the context and secret key), then indeed the comple…
For a given short-context (hash function seeding) you do have detectable manipulation of the completion (how they can read the marking). But, because (assuming enough entropy in context) the hashing function is decoupled from the log-probs, the perturbations are independent from the underlying distribution, so you're still sampling from the same distribution quotient some noise. The only way you'd notice this is they…
Anyway, there are lots of cases where text carries very little entropy. E.g. boilerplate code, exchanges of pleasantries, well-worn platitudes and jokes, etc. These are sequences of tokens that will be seen across many, many separate outputs. Watermarking here (on the token following the common sequence) would thus be easily detectable and noticed as a claude style. The longer the hashing context though, the lower the amount of pathological cases with low entropy. It would be interesting to understand the exact parametrization better!
Earlier quoted context omitted.
> Neither can I recall there has ever been a moveable type press, Ah my friend, you are about to fall down a rabbit hole into standardisation of spelling, and the sometimes deadly debates about how to translate latin into the vernacular. English, as she is written, is a great example. for the spoken word, BBC/received pronunciation is another. I speak the way I do _because_ of BBC radio. The reason I have the accent…
A radio device at BBC did all this and not humans? > again, your argument is against LLMs and globalisation of culture, not finger printing. ABSOLUTELY NOT. Do not ever put words in my mouth. My argument is that radio is a medium through which humans communicate. An LLM is not.
In the same way that handwriting conveys more information about the writer than type, typing ones own thoughts conveys more information about the writer than prompting an LLM.
The analogy here is hiring a speech writer to do your speeches, or dictating to a skilled typist.
I understand the vociferousness in push back
Earlier quoted context omitted.
I think the author is just mad people will be able to detect and filter out their AI slop writing in the future.
Gruber has long been a talented, excellent writer. I hugely doubt he uses AI at all, nor does he plan to. His first take on this situation was cutely naive, thinking they were going to inject secret hidden unicode characters. But ultimately he has a massive hate on for the EU -- they were mean to Apple once -- and it comes out in any topic that overlaps.
He notes in various other posts that he uses AI/LLMs and chatbots quite extensively. (I don't recall what for exactly, but not for writing his pieces.)