Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code is steganographically marking requests

thereallo.dev

Post reply on HN
Print view View original

401–410 of 817 posts

Re: Claude Code is steganographically marking requests

#401

Post by chatmasta »

superfrank wrote: post #164
meowface wrote: post #92

Value judgment aside: I am a bit surprised at how sloppily they did this. I think they could've achieved the same effect while decreasing the odds of detection via reverse engineering. (This field is known as "underhanded code", coined by the Underhanded C contest: https://www.underhanded-c.org . It's a little-known "art"; little-known for probably self-explanatory reasons. There are much cleverer ways of achieving o…

It's also possible that there are more in-depth detection methods and that this was just a cheap and easy first step that hasn't been removed because it catches a lot of less sophisticated bad actors. It's unlikely that this will stop a big AI lab from distilling their model if they're really determined, but A) it may be enough to stop a bunch of fly-by-night token resellers looking to make a quick buck and B) you ne…

It just needs to work for a few days after bundle release before the mice find out where the cat is hiding. By then it’s too late, the cat already sees the paw prints and droppings into the mouse hole.

Re: Claude Code is steganographically marking requests

#402

Post by meowface »

chatmasta wrote: post #401
superfrank wrote: post #164

Earlier quoted context omitted.

It's also possible that there are more in-depth detection methods and that this was just a cheap and easy first step that hasn't been removed because it catches a lot of less sophisticated bad actors. It's unlikely that this will stop a big AI lab from distilling their model if they're really determined, but A) it may be enough to stop a bunch of fly-by-night token resellers looking to make a quick buck and B) you ne…

It just needs to work for a few days after bundle release before the mice find out where the cat is hiding. By then it’s too late, the cat already sees the paw prints and droppings into the mouse hole.

It is quite possible this was intended as a "fast burn" measure, yeah

Re: Claude Code is steganographically marking requests

#403

Post by orbital-decay »

solenoid0937 wrote: post #380
orbital-decay wrote: post #357

Earlier quoted context omitted.

The purpose of system is what it does. Can you read their previous musings about the glorious future, look at what they actually do, read Amodei's batshit insane nationalistic rants, and say in all seriousness yeah it's the kind of people I want to entrust my entire future life? >you want to prevent China from getting to superintelligence first I don't. Prevent , not even outpace? Why? Seems like you're assuming Chin…

[flagged]

>Please actually do a modicum of research into AI safety. Your comment is the equivalent of a patient with zero context, arguing against the position of established medical science.

What makes you think I didn't? You're talking like it's self-evident and adopt the condescending tone from the start, without giving any actual arguments why. (I'm not really interested in them as all these discussions are pointless and we had them back in ~2015)

>A "cult" implies belief in something unknowable/unprovable.

Yes, precisely. Also the gods and religious practices. Rationalists and subsequently AI safety branch invented a religion in a roundabout way.

>"The entire medical community was born in medical circles and it's a very specific cult"

Medicine is largely based on evidence and real-life observations, unlike AI safety which is based on belief in something that doesn't exist and some unprovable lore that is entirely rationalized without any grounding, and is expected to be self-evident (because it obviously is) and believed by the others. One is science, another is policy.

>Are you familiar with the history of the PRC?

Yes, I know it extremely well. I also know the history of the US, am familiar with the people who do AI research in the US from before they started doing this, and can see the actual reality.

Re: Claude Code is steganographically marking requests

#404

Post by gjvc »

botfriendsarent wrote: post #304
gleenn wrote: post #279

Earlier quoted context omitted.

Whether or not you find Anthropic's behavior bad, theybhave been very loudly stating the foreign labs have been distilling their models for a while now. This seems like an obvious response to me that would be a mechanism to make that obvious.

Sounds suspiciously similar to the "album title", "Steal this album" by system of a down. Im not sure why we are dithering on the boundaries of honesty when the entire content LLMs are trained on is stolen. Are we debating "honor among thieves"? Of course we are not, or maybe we are! Does the behavior of a thief even matter to me? only after they do their time. And they will. I can see the investors perched on the ba…

"Steal this book" by Abbie Hoffman

Re: Claude Code is steganographically marking requests

#405

Post by karaziox »

dools wrote: post #309
mrshadowgoose wrote: post #215

The conclusion of this blog post is a bit hysterical. The intent of this steg is excruciatingly clear (identifying usage by Chinese firms that may be conducting model distillation). It's unclear on how this "punishes normal developers" in any shape or form.

Why would a Chinese firm distilling the product use Claude code?

They offer (extremely) discounted Claude prices but you have to go through their gateway. They subsidize part of that, and they get the low price by reselling unused Max capacity, there's been a few posts on that in the past months. People are apparently getting 90% discounts on their claude use this way, tradeoff is that you have two companies learning from your data, instead of just one. So people use the same tools they use normally, but get it for a lot cheaper

Re: Claude Code is steganographically marking requests

#406

Post by ozozozd »

civet_java wrote: post #238

There are some commentors in this thread downplaying the severity of a service provider being less than transparent about exactly what their shipped tooling does on customer's machines. That the provider's business needs necessitate the this behaviour doesn't justify their lack of honest disclosure. That honest disclosure would render the solution to their problem useless isn't my problem. If anything, that they thou…

They are the chosen ones. Protecting the world from

Any and all ends justify any and all means.

/s

Re: Claude Code is steganographically marking requests

#407

Post by solenoid0937 »

orbital-decay wrote: post #403
solenoid0937 wrote: post #380

Earlier quoted context omitted.

[flagged]

>Please actually do a modicum of research into AI safety. Your comment is the equivalent of a patient with zero context, arguing against the position of established medical science. What makes you think I didn't? You're talking like it's self-evident and adopt the condescending tone from the start, without giving any actual arguments why. (I'm not really interested in them as all these discussions are pointless and w…

> Rationalists and subsequently AI safety branch invented a religion in a roundabout way

If you are arguing in good faith you can very clearly reason about any given AI safety take. Case in point, you refused to engage with most of the questions because you know the conclusions they lead to.

> Medicine is largely based on evidence and real-life observations, unlike AI safety

"AI safety doesn't exist" is certainly a take.

> Yes, I know it extremely well. I also know the history of the US and see the actual reality.

Why do you think it's better that a country that turns its citizens into a pulp for criticizing the government, and censors most media to control its citizens' thoughts, reach SI before one that is democratically elected and in which you can generally criticize the government?

Re: Claude Code is steganographically marking requests

#408

Post by KoftaBob »

gleenn wrote: post #279
civet_java wrote: post #238

There are some commentors in this thread downplaying the severity of a service provider being less than transparent about exactly what their shipped tooling does on customer's machines. That the provider's business needs necessitate the this behaviour doesn't justify their lack of honest disclosure. That honest disclosure would render the solution to their problem useless isn't my problem. If anything, that they thou…

Whether or not you find Anthropic's behavior bad, theybhave been very loudly stating the foreign labs have been distilling their models for a while now. This seems like an obvious response to me that would be a mechanism to make that obvious.

> a mechanism to make that obvious.

Say they prove that foreign labs are distilling their models, then what?

Re: Claude Code is steganographically marking requests

#409

Post by orbital-decay »

solenoid0937 wrote: post #389
Biganon wrote: post #387

Earlier quoted context omitted.

How does it lead to the end of humanity if China reaches it first? You really think the US is more trustworthy?

What are the PRC's values? How are individual freedoms in China? What happens if you criticize the government as a Chinese citizen? Is it a good thing if a government that turns its citizens into red pulp for criticism, or disappears them in the middle of the night, or bans access to most media, is the first to a godlike superintelligence that gives them de-facto control of (and impose their values upon) the whole wo…

Can you substitute PRC/China for Anthropic and try answering your questions?

Re: Claude Code is steganographically marking requests

#410

Post by solenoid0937 »

orbital-decay wrote: post #409
solenoid0937 wrote: post #389

Earlier quoted context omitted.

What are the PRC's values? How are individual freedoms in China? What happens if you criticize the government as a Chinese citizen? Is it a good thing if a government that turns its citizens into red pulp for criticism, or disappears them in the middle of the night, or bans access to most media, is the first to a godlike superintelligence that gives them de-facto control of (and impose their values upon) the whole wo…

Can you substitute PRC/China for Anthropic and try answering your questions?

No company will own superintelligence. Governments will, just like governments own nuclear weapons (developed by companies).

So the comparison is with the US, not Anthropic.

The US doesn't turn its citizens into a fine red purée for criticizing it.

The US doesn't censor most media.

It is strictly better for a democratic nation like the US to get to superintelligence before a country that will gladly blend its citizens for criticizing it, and censor anything that dares to challenge its power.

Post reply on HN