Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

daringfireball.net

Post reply on HN
Print view View original

671–680 of 776 posts

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#671

Post by Fogest »

gwd wrote: post #243
beering wrote: post #85

Earlier quoted context omitted.

Google has A/B tested watermarking on millions of responses. They say they observed no difference in user behavior.

It says they observed no difference in people clicking thumbs up or down. There are loads of other behavior that they didn't observe; like, say, switching to a different LLM.

Agreed, if I simply didn't like the style or words an AI was using in something it wrote, I would switch to a competitors and see what it can come with. I probably wouldn't hit the thumbs down on the Gemini response as it's not that the response is wrong, I just didn't like it. I usually reserve the thumb down for when the AI is wrong.

Also, depending on what I am asking it, I often don't want to use the thumb down or up, as this may mean my conversation is going to have some kind of human review and depending on what I am asking for, I may not want to bring attention to my stuff.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#672

Post by einpoklum »

Lumich wrote: post #575
einpoklum wrote: post #401

Earlier quoted context omitted.

> (electricity prices for instance) because nobody can agree on anything. I would say that's more like because the US has arranged for Europe's fossil fuel energy sources to be disrupted or cut off: * Libya - NATO made a pig's breakfast of that, it's a failed state now. * Iran - transitive sanctions, because why not prevent non-US states from trading with each other. * Russia (& Kazahkhstan) - The US (with or without…

Spot on. Also, in 2003, the war on Iraq, still occupied. And the proxy war on Syria (stifling an unwelcome pipeline project). European “leaders” pretend to not comprehend how they're being screwed. Stockholm syndrome. Populations don't understand, propaganda (“free press”) working correctly.

It's very weird looking in from the outside. I mean, sure, the US is the dominant power and everything, so things like Iraq and Iran could be construed as just collateral damage from their imperial maneuvers. But it has just piled on, more and more, and even when they are hit right in the face with the massive bombing of NordStream, still practically nobody tries to draw any sort of line.

Reminds me a bit of that 'Yes Prime Minister' sketch about nuclear deterrant, when the skeptical conversant asks the PM: "So what is the last resort, Picadilli?"

https://www.youtube.com/watch?v=IX_d_vMKswE

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#673

Post by matheusmoreira »

josephg wrote: post #652
matheusmoreira wrote: post #482

Earlier quoted context omitted.

> The people who cheat miss out on learning. They aren't there to learn. They are there to jump through hoops to get a degree that will let them get a job so they can make money and prosper . The learning is entirely secondary. The cheating will stop when there is no longer any economic incentive to be there in the first place. People with "pure" motives will refuse to cheat on their own, precisely because they want…

Got any evidence for that claim? I’ve worked with plenty of smart, self taught programmers throughout my career. The highest paid guy I know didn’t finish high school.

> Got any evidence for that claim?

Bureau of Labor Statistics. Jobs requiring higher education pay roughly 2x more than those requiring high school education and roughly 3x more than those requiring no education at all.

And even if there's no formal requirement for a degree, it doesn't automatically mean people lacking degrees will get hired either.

Anyone who wants to get a well paying white collar job pretty much needs a degree.

> I’ve worked with plenty of smart, self taught programmers throughout my career.

And how did they get the job? Networking?

> The highest paid guy I know didn’t finish high school.

I think this ought to be the rule, not the exception.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#674

Post by alphazard »

Humans have been inserting benign noise into their writing without affecting the signal for thousands of years. They called it "style".

Apparently superfluous descriptions, unnecessary words, and paragraph footnotes are all fine, but gumbel softmax (or whatever's going on here) isn't.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#675

Post by chancedurham »

mrngld wrote: post #404
Topfi wrote: post #377

Earlier quoted context omitted.

Fair point, for text it is far harder to prevent signatures being applied to generated text vs images at the moment of capture and most approaches I can come up with to remedy this can either be bypassed (edit histories can be output by models similar to humans) or will be controversial. Taking a page out of the anti-cheat textbook, mainly written for gaming, there are methods which might hold in the medium term. Les…

I've been thinking for a while that all of this is just trying to grasp tighter the last bits of sand escaping between our fingers. The end game, perhaps, is trust. Do you trust or know the source? If you don't, assume it was AI generated. If you do, accept it as authentic based on whatever they disclose, but know that it's possible they aren't being totally honest or were themselves fooled in some way, depending on…

[flagged]

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#676

Post by out_of_protocol »

unbelievr wrote: post #361
ThePhysicist wrote: post #338

Earlier quoted context omitted.

Would really like to know how their watermarking technique works, and if they can use it to store arbitrary information in the text (I assume they can if only in a limited way). I assume if the text is long enough they could add all kinds of metadata that would then be undetectable as the model that generated the text is the "cryptographic" key that encodes the data. I wonder if you can have another model run over th…

The paper for it is open. The technique isn't really hiding information in the text itself, but by forcing some of the rolls to follow a specific pattern. LLMs work by estimating the most likely next token, so there's sometimes a list of possible candidates that would all work in the text (e.g. synonyms). At low "temperature", the output is a bit more deterministic and otherwise it's a weighted dice roll of which tok…

Does that also mean owner must re-run very model ever released by them to detect anything?

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#677

Post by tarvaina »

ucha wrote: post #650

> “By definition it must make text worse … because the nature of the watermarking algorithm requires it to sometimes increase the probability of selecting a worse word choice and decrease the probability of selecting the model’s best choice.” Gruber made an effort to but doesn't fully understand how SynthID works. LLMs select the next word randomly from a set probability distribution, so there is no "best choice" unl…

I came here to quote the same sentence. Here's another way to look at it:

Suppose there actually is a best word choice. The LLM doesn't know what it is but makes a guess. Maybe it's the best one, maybe it isn't. The probability that SynthID changes the best choice to a worse one is equal to the probability that it changes a worse choice to the best one.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#678

Post by ozozozd »

cratermoon wrote: post #570
ThePhysicist wrote: post #338

Earlier quoted context omitted.

Would really like to know how their watermarking technique works, and if they can use it to store arbitrary information in the text (I assume they can if only in a limited way). I assume if the text is long enough they could add all kinds of metadata that would then be undetectable as the model that generated the text is the "cryptographic" key that encodes the data. I wonder if you can have another model run over th…

I realize that short attention spans are pervasive now, but the link to the explanation is only eight paragraphs in https://declaude.org/watermarking/

Yeah kind of surprised that people didn’t have enough patience for Gruber’s writing.

It’s quite fun and engaging.

Re: Anthropic's ‘watermark’ text adulteration in Claude is a perversion of writing

#679

Post by josephg »

matheusmoreira wrote: post #673
josephg wrote: post #652

Earlier quoted context omitted.

Got any evidence for that claim? I’ve worked with plenty of smart, self taught programmers throughout my career. The highest paid guy I know didn’t finish high school.

> Got any evidence for that claim? Bureau of Labor Statistics. Jobs requiring higher education pay roughly 2x more than those requiring high school education and roughly 3x more than those requiring no education at all. And even if there's no formal requirement for a degree, it doesn't automatically mean people lacking degrees will get hired either. Anyone who wants to get a well paying white collar job pretty much n…

> Jobs requiring higher education pay roughly 2x more than those requiring high school education

This isn’t really evidence either way. Why do companies pay twice as much for people with higher education? We can’t tell from that statistic. Maybe it’s what you learn in class that makes you twice as valuable to potential employers.

> how did they get the job? Networking?

Probably. After all, that’s how most people in our industry find work. Degree or not.

Post reply on HN