Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code is steganographically marking requests

thereallo.dev

Post reply on HN
Print view View original

531–540 of 817 posts

Re: Claude Code is steganographically marking requests

#531

Post by bayindirh »

CMay wrote: post #509
bayindirh wrote: post #291

Earlier quoted context omitted.

From my understanding, distilling the model with another model is not illegal per se. Also, the output of the LLM is public domain by law, too. So, why all this "effort" to protect the model? This is a free market, and moving fast and breaking things is the norm. If they are so adamant on protecting their IP, maybe they can start by respecting others' IP, so we can start talking about ethics, equality and playing fai…

The usage of the output is probably considered legal. The usage of the service for that purpose may not be, and using it at scale in a dishonest way is not, which is what China has been doing. Countless thousands of separate requests abusing the service (which is not a simple static HTML feed, but an AI service request) for every kind of query to soak up the results. The post is about what's in the local code, but fo…

> Why does it matter if China gains those capabilities? I invite you to begin to learn about China's behavior around the world. The CCP is darkside material.

Reminds me of this comic: https://xcancel.com/tomgauld/status/571994690289061888?lang=...

None of the superpowers in this world is innocent, and like MAD, more countries have the capability, the better.

I know some of the things CCP do/did. I know some of the things US does/did. I'm from neither, so I don't take sides.

AI's use has been confirmed, or more precisely boasted by two countries in two different wars, and China was not one of these countries.

We have seen the effects of "if they don't know them, they can't exploit them" mindset of NSA for years. Keeping information/technology private is neither beneficial, nor possible. It's only a temporary moat-ish gap. Not a definitive solution.

Re: Claude Code is steganographically marking requests

#532

Post by piokoch »

Jimmc414 wrote: post #512
civet_java wrote: post #238

There are some commentors in this thread downplaying the severity of a service provider being less than transparent about exactly what their shipped tooling does on customer's machines. That the provider's business needs necessitate the this behaviour doesn't justify their lack of honest disclosure. That honest disclosure would render the solution to their problem useless isn't my problem. If anything, that they thou…

I don't see any ethical connection between adding canary tokens to your output to catch people breaking your accepted ToS through ongoing distillation and stealing your PII off of your machine. How are legitimate users in US or China possibly harmed by Anthropic silently changing the apostrophe in Today's or the date separator from - to /?

You are talking about "stealing" from people who already used stolen texts to train their models.

Re: Claude Code is steganographically marking requests

#533

Post by throwawayffffas »

civet_java wrote: post #238

There are some commentors in this thread downplaying the severity of a service provider being less than transparent about exactly what their shipped tooling does on customer's machines. That the provider's business needs necessitate the this behaviour doesn't justify their lack of honest disclosure. That honest disclosure would render the solution to their problem useless isn't my problem. If anything, that they thou…

Not only that, undisclosed behavior of this nature erodes the trust that the software is not compromised by internal bad actors.

Sure the thing we know matches the company interests but as the parent mentions for all we know they are also shipping over your ssh keys and browser cookies.

Re: Claude Code is steganographically marking requests

#534

Post by throwawayffffas »

FooBarWidget wrote: post #530
kiproping wrote: post #284

Earlier quoted context omitted.

First its the "Chinese" then it will be people using "cyber" capabilities, or "jailbreaking" or "going against Dario" or any other thing they find "objectionable".

Their terms of service already effectively attacks people for criticizing Anthropic. It says that if you use Claude to criticize Anthropic, then you've pre-agreed to pay for their lawyers going after you, and pre-agreed to lose the court case.

Since Fable if you use Claude to do ML research they find objectionable you are delegated to less capable models.

Re: Claude Code is steganographically marking requests

#535

Post by bayindirh »

stale2002 wrote: post #457
bayindirh wrote: post #291

Earlier quoted context omitted.

From my understanding, distilling the model with another model is not illegal per se. Also, the output of the LLM is public domain by law, too. So, why all this "effort" to protect the model? This is a free market, and moving fast and breaking things is the norm. If they are so adamant on protecting their IP, maybe they can start by respecting others' IP, so we can start talking about ethics, equality and playing fai…

> So, why all this "effort" to protect the model? Because it's their model and business and they are free to use the free market to do exactly that? That's their free market rights too. If you don't like it, use another model (which they would be fine with).

> Because it's their model and business and they are free to use the free market to do exactly that?

I mean, nothing stops distillers to find better ways to distill, either. Meaningless cat & mouse games.

> If you don't like it, use another model (which they would be fine with).

Thanks, I use none. It's peaceful this way.

Re: Claude Code is steganographically marking requests

#537

Post by jchonphoenix »

meowface wrote: post #92

Value judgment aside: I am a bit surprised at how sloppily they did this. I think they could've achieved the same effect while decreasing the odds of detection via reverse engineering. (This field is known as "underhanded code", coined by the Underhanded C contest: https://www.underhanded-c.org . It's a little-known "art"; little-known for probably self-explanatory reasons. There are much cleverer ways of achieving o…

It was likely done by claude

Re: Claude Code is steganographically marking requests

#539

Post by bot403 »

dackdel wrote: post #500
ok123456 wrote: post #310

Earlier quoted context omitted.

Won't somebody please think of the Chinese cyber children!

what about the "ai children who are hosted on servers in china and usa" think about them too

I asked Claude to think of the children. It worked for three minutes, came up with a plan, and charged me $9.

Re: Claude Code is steganographically marking requests

#540

Post by eloisius »

ToValueFunfetti wrote: post #499
eloisius wrote: post #492

Earlier quoted context omitted.

I think Anthropic will argue whatever argument is likely to protect their interests. I don’t expect anything consistent or moral from them. My quibble is with all the Anthropic fanboys who repeat this crap.

I'd think the problem with fanboys is that they don't care about the truth of the underlying arguments. They just want to score points for their team. Do you have a different issue with them? If not, why not engage with the arguments yourself?

I know from reflecting on my own beliefs now compared with 10-15 years past that one's beliefs can change, and I don't want to be so cynical as to say that these fanboys don't actually believe what they are saying and only want to score points. I'm sure there's a great deal of commentary that is astroturf, but I think there are plenty of (hopefully young and naive) techno-optimists who sincerely think companies like Anthropic can do no wrong and only move humanity forward, or something like that.

In any case, online debate is not always about changing the mind of the single person you engaged with. To some degree, its performative debate so that other readers may be influenced by your ideas.

Post reply on HN