Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code is steganographically marking requests

thereallo.dev

Post reply on HN
Print view View original

581–590 of 817 posts

Re: Claude Code is steganographically marking requests

#581

Post by actionfromafar »

ammo1662 wrote: post #544
civet_java wrote: post #238

There are some commentors in this thread downplaying the severity of a service provider being less than transparent about exactly what their shipped tooling does on customer's machines. That the provider's business needs necessitate the this behaviour doesn't justify their lack of honest disclosure. That honest disclosure would render the solution to their problem useless isn't my problem. If anything, that they thou…

This is not a technical issue or a matter of service agreements. They totally knew that this would never be accepted by users, and that is precisely why they resorted to obfuscation and steganography to exfiltrate the data. This was quietly added in an update, and would have been removed in a later one had it gone unnoticed. How is this any different from hiding a drug in food and randomly feeding it to a homeless pe…

> randomly feeding it to a homeless person as a human trial

Would you be shocked if some of the current tech bro companies could with finding such subjects?

Re: Claude Code is steganographically marking requests

#582

Post by CMay »

bayindirh wrote: post #531
CMay wrote: post #509

Earlier quoted context omitted.

The usage of the output is probably considered legal. The usage of the service for that purpose may not be, and using it at scale in a dishonest way is not, which is what China has been doing. Countless thousands of separate requests abusing the service (which is not a simple static HTML feed, but an AI service request) for every kind of query to soak up the results. The post is about what's in the local code, but fo…

> Why does it matter if China gains those capabilities? I invite you to begin to learn about China's behavior around the world. The CCP is darkside material. Reminds me of this comic: https://xcancel.com/tomgauld/status/571994690289061888?lang=... None of the superpowers in this world is innocent, and like MAD, more countries have the capability, the better. I know some of the things CCP do/did. I know some of the th…

Certainly the world is full of actions and reactions, nothing is happening in a vacuum. You don't have to be from a country to take sides, but presumably you have some kind of moral compass, some kind of values around personal freedom or the worth of a human life.

There can be a very real cost, because one side comes from an ideology with a history that wants to conquer the entire Earth which caused World War 2 while the other side is trying to prune the planet like a bonsai to prevent it from descending into total chaos to preserve some sense of international order.

Europe was constantly at war, and we helped stabilize it. Middle East as been constantly at war, and if Iran can be sorted then it will be the closest to some sense of peace it's been in a long time.

We used to be in Japan, Philippines, Germany, Vietnam, South Korea, Iraq, Afghanistan and so on. How many are US territories? None. We aren't out there to conquer the globe and take land. We're usually fighting other people's wars for them, because they're up against better resourced opponents. Meanwhile China is over there building artificial islands, ramming other country's ships, creating ideological police stations in countries around the world to harass people and engaging in the most widespread international interference campaigns in human history.

They do not treat their people well and they do not have free speech. The internet is flooded with their propaganda now, because they have a human numbers advantage.

It's true that given time most advantages are temporary, but there's always that slim chance we could slow them down until the CCP collapses and they could become a more normal country.

Re: Claude Code is steganographically marking requests

#583

Post by actionfromafar »

plasticeagle wrote: post #562
kalkin wrote: post #471

Earlier quoted context omitted.

> The code is not eligible for copyright. This is very much not what the linked case established.

According to the link: "The US Copyright Office and federal courts require human authorship for copyright protection; works created solely by AI are not eligible for registration under the current rules." The Supreme Court declined to consider a challenge to this rule, and so for the moment at least, the rule remains in place. This means that companies leaning heavily into their LLM use may very well find that they d…

I think the word "solely" is going to be a tunnel you can drive freight trains through.

So with the asbestos analogy, we encase the fibers in resin and call the whole thing copyrighted.

Re: Claude Code is steganographically marking requests

#584

Post by ed_elliott_asc »

“ and push commits”

Am I the only person who insists on writing my password every time I push and pull from git?

Originally I didn’t want IDE’s doing stuff for me, now I absolutely do not want an LLM to have that power.

Is it really that unique to control what git does remotely?

Re: Claude Code is steganographically marking requests

#585

Post by tripzilch »

next_xibalba wrote: post #302
cowl wrote: post #295

Earlier quoted context omitted.

oh no, the company that illegally used every possible media they could get their hands on is crying that some other company is doing something potentially shady but not illegal? And using that excuse to put in place hidden surveillance systems on their customers?

People keep throwing this idea around haphazardly, but U.S. courts have pretty consistently decided that training on copyrighted works falls under fair use. You may not like it, but that doesn't make it "illegal".

Maybe "U.S. courts have pretty consistently decided" used to mean something, but I don't think the opinion of US courts should be the standard for anything, anymore.

Re: Claude Code is steganographically marking requests

#586

Post by lelanthran »

theshrike79 wrote: post #553
lelanthran wrote: post #491

Earlier quoted context omitted.

> probably related to protecting their IP The same IP that is a highly compressed collection of everyone's else's IP? That's hilarious.

If some other AI company wants a highly compressed collection of everyone else's IP, they can get it by themselves - not from Anthropic pre-compressed =)

Actually, no, I think I'll rather just take it from them.

Re: Claude Code is steganographically marking requests

#587

Post by jwrallie »

ed_elliott_asc wrote: post #584

“ and push commits” Am I the only person who insists on writing my password every time I push and pull from git? Originally I didn’t want IDE’s doing stuff for me, now I absolutely do not want an LLM to have that power. Is it really that unique to control what git does remotely?

You are not alone, for me committing something it means I am signing my responsibility for it. I may not type a password, but I am always the one pressing the enter key.

Re: Claude Code is steganographically marking requests

#588

Post by Gareth321 »

Grimblewald wrote: post #494
mrshadowgoose wrote: post #215

The conclusion of this blog post is a bit hysterical. The intent of this steg is excruciatingly clear (identifying usage by Chinese firms that may be conducting model distillation). It's unclear on how this "punishes normal developers" in any shape or form.

So block people, instead of having false positives be secretly fucked over, and having them pay for the pleasure? Given the hidden model degradation of fable and now this, what makes you think this is where it stops? That's just what we know about and there's clearly a long-standing and deeply rooted malicious intent here. I've had Claude fuck over clean well documented code-bases for no reason, and there's a good ch…

> So block people

To be fair to Anthropic, [they're trying very hard to do that.](https://www.anthropic.com/news/detecting-and-preventing-dist...). The attacks are sophisticated and difficult to detect. I don't accept that if this fails, their only option is to just accept Chinese companies stealing IP.

Re: Claude Code is steganographically marking requests

#589

Post by jonathanstrange »

filoleg wrote: post #311
bayindirh wrote: post #291

Earlier quoted context omitted.

From my understanding, distilling the model with another model is not illegal per se. Also, the output of the LLM is public domain by law, too. So, why all this "effort" to protect the model? This is a free market, and moving fast and breaking things is the norm. If they are so adamant on protecting their IP, maybe they can start by respecting others' IP, so we can start talking about ethics, equality and playing fai…

> distilling the model with another model is not illegal per se. Just because it is legal, that doesn't mean Anthropic wouldn't reasonably want to prevent that from happening (which, from my understanding, isn't illegal either).

What Anthropic is doing is illegal in many jurisdictions. I don't know about the legal situation for the Chinese domains they mark, but steganographic data extraction without user consent would definitely be illegal in the EU, for example.

Re: Claude Code is steganographically marking requests

#590

Post by Davidzheng »

nomel wrote: post #244
Terr_ wrote: post #222

Earlier quoted context omitted.

> hysterical. The intent of this steg is excruciatingly clear Even good goals do not excuse malicious or reckless execution. The ends do not always justify the means. Whether or not it harmed you this time , it's a violation of trust and autonomy. Surely you'd be angry if someone secretly installed a rootkit onto your computer, even if--at least for now--it only had code to try to detect and snitch on Public Enemy #1…

What do you see as malicious or reckless here, exactly? This seems to be a VERY low resolution, functionally anonymous, bit of info, probably related to protecting their IP from bad actors breaking the TOS. This looks like it's covered in the second bullet point of the "Personal data we automatically receive", that you consented to: > Usage Information: We collect information about your use of the Services, such as t…

Is it completely clear to you what the purpose of this is? It isn't completely clear to me but it's very likely it's an issue with me. I feel that it can be part of some larger counteroffensive against certain actors in China in a way that is more than just the signalling here--like maybe there's much more we haven't seen. In any case, it certainly shows their willingness to use less conventional tactics against those they view as adversaries.
Post reply on HN