Live data from Hacker News

Claude Code is steganographically marking requests

thereallo.dev

781–790 of 817 posts

Re: Claude Code is steganographically marking requests

#781

Earlier quoted context omitted.

I disagree with your assessment that large organisations are beneficial. We can see with our current crop of large organisations that they really struggle to create anything new; most of their new products or services were developed by a small organisation and then acquired. A lot of those products are then enshittified and badly managed because large organisation politics screws things up. Large organisations are in…

And I disagree with yours :). Large organizations are necessary if you want things like airplanes and rockets and computers and MRI machines to exist. And if you feel you benefit from those things yourself, then large organizations that create and operate[0] them are beneficial to you, too. > A lot of those products are then enshittified and badly managed because large organisation politics screws things up. That's n…

I actually agree with most of this. A few points:

SpaceX built great rockets before it became large (though this is relative - a small rocket company is a large hairdresser, for example). There is a certain scale required for some types of business, agreed. But getting larger doesn't necessarily make them better.

> That description fits small business owners much better IMO. In our times, at least in non-failed western countries, there's a limit to how abusive or careless a large organization can be with their customers or employees - their very size makes them easy to target legally. It might be hard to get through their well-funded legal defense, unless the case is slam dunk, but that's still much better than the armies of small businesses flying completely under the radar, flagrantly violating basic health and safety regulations, or flat out lying to customers in their face, because they're not worth the effort of investigating.

Flat disagree with this. Small org CEOs are close to their customers and employees and if they behave like dicks then they get punished quickly. Obviously some still do, because people, but it's harder for a small company CEO to continue being a dick.

> Anyway, key point: *there is no categorical difference between "large organizations" and "small organizations". You need a certain amount of people and communication (and capital) to do high-complexity endeavors. The difference between a well-integrated big corporation, and a hundred of small businesses that kinda end up together delivering something big, is just that the latter is using the market as management layer.

There is a key step change when the first pure-management layer forms in an organisation. This is the management layer that only have other managers reporting to them, and only report to other managers. So no direct contact with front-line staff or shareholders. Personally, the presence of this layer is what classifies an organisation as "large". It's when the politics takes over from performance as the priority and the organisation starts to lose the connection between what the c-suite want and what the front-line actually do.

And all commercial airplanes, MRIs, anything, were built first by small organisations, and only later by large orgs. Large orgs just can't invent new things unless they form specialist small orgs to do it (skunkworks, or Palo Alto, or similar). Large orgs just don't work like that.

Re: Claude Code is steganographically marking requests

#782
post #582

Earlier quoted context omitted.

> Why does it matter if China gains those capabilities? I invite you to begin to learn about China's behavior around the world. The CCP is darkside material. Reminds me of this comic: https://xcancel.com/tomgauld/status/571994690289061888?lang=... None of the superpowers in this world is innocent, and like MAD, more countries have the capability, the better. I know some of the things CCP do/did. I know some of the th…

Certainly the world is full of actions and reactions, nothing is happening in a vacuum. You don't have to be from a country to take sides, but presumably you have some kind of moral compass, some kind of values around personal freedom or the worth of a human life. There can be a very real cost, because one side comes from an ideology with a history that wants to conquer the entire Earth which caused World War 2 while…

Ah yes. The exceptionalism argument.

There's the good "us" and the bad "them."

Re: Claude Code is steganographically marking requests

#783

Earlier quoted context omitted.

So block people, instead of having false positives be secretly fucked over, and having them pay for the pleasure? Given the hidden model degradation of fable and now this, what makes you think this is where it stops? That's just what we know about and there's clearly a long-standing and deeply rooted malicious intent here. I've had Claude fuck over clean well documented code-bases for no reason, and there's a good ch…

> So block people To be fair to Anthropic, [they're trying very hard to do that.]( https://www.anthropic.com/news/detecting-and-preventing-dist... ). The attacks are sophisticated and difficult to detect. I don't accept that if this fails, their only option is to just accept Chinese companies stealing IP.

They're not "attacks." Anthropic calling them "attacks" doesn't make them attacks. Companies are collecting transcripts of conversations with Anthropic, and giving users a discount to share them.

As many have pointed out, they're collecting data in ways much less aggressive than Anthropic itself about what Anthropic does.

Anthropic doesn't like it, but I don't see this as "Chinese companies stealing IP," any more than if Google tried to ban competitors from seeing how Google Docs or an Android phone behaves, or Ford trying to ban anyone from Toyota from seeing what their car looks like.

Please stop calling them "attacks." It's distillation training. It's looking at what Anthropic does -- as a block box -- and trying to duplicate or beat it. It's how progress happens.

Re: Claude Code is steganographically marking requests

#784

Earlier quoted context omitted.

I disagree with your assessment that large organisations are beneficial. We can see with our current crop of large organisations that they really struggle to create anything new; most of their new products or services were developed by a small organisation and then acquired. A lot of those products are then enshittified and badly managed because large organisation politics screws things up. Large organisations are in…

They are beneficial for those with an equity stake . That much is clear.

[deleted]

Re: Claude Code is steganographically marking requests

#785
post #582

Earlier quoted context omitted.

Certainly the world is full of actions and reactions, nothing is happening in a vacuum. You don't have to be from a country to take sides, but presumably you have some kind of moral compass, some kind of values around personal freedom or the worth of a human life. There can be a very real cost, because one side comes from an ideology with a history that wants to conquer the entire Earth which caused World War 2 while…

Ah yes. The exceptionalism argument. There's the good "us" and the bad "them."

I didn't make an exceptionalist argument, but if any country's behavior and values can be measured compared to others, you will always be able to make some kind of decision about where those fall in terms of goodness or badness. Do you not believe in good and bad?

Re: Claude Code is steganographically marking requests

#786

Earlier quoted context omitted.

It's also possible that there are more in-depth detection methods and that this was just a cheap and easy first step that hasn't been removed because it catches a lot of less sophisticated bad actors. It's unlikely that this will stop a big AI lab from distilling their model if they're really determined, but A) it may be enough to stop a bunch of fly-by-night token resellers looking to make a quick buck and B) you ne…

But what's stopping someone from modifying or injecting code into the client to bypass this 'restriction'? This type of security should be implemented server-side and at the network perimeter not in the client, especially one written in TypeScript.

I think the idea is that _regular_ people will use CC client but via a 3rd party reseller, and the reseller intercepts the data for distilling (or anything). Requiring the users to use modified clients, or really do anything more than simply change the service URL, would add much more friction for people just looking to use CC for cheaper

Re: Claude Code is steganographically marking requests

#787

Earlier quoted context omitted.

> I love the asymmetry. Much as I hate to defend companies climbing to success and pulling up the ladder afterwards, this asymmetry you note is kind of the whole point a company would want to grow big . Growing an organization has some super-linear costs and generally sucks for most individuals living through it - including the management - but it's still considered worth it, precisely because big entities can do thi…

I disagree with your assessment that large organisations are beneficial. We can see with our current crop of large organisations that they really struggle to create anything new; most of their new products or services were developed by a small organisation and then acquired. A lot of those products are then enshittified and badly managed because large organisation politics screws things up. Large organisations are in…

[deleted]

Re: Claude Code is steganographically marking requests

#788

Earlier quoted context omitted.

And I disagree with yours :). Large organizations are necessary if you want things like airplanes and rockets and computers and MRI machines to exist. And if you feel you benefit from those things yourself, then large organizations that create and operate[0] them are beneficial to you, too. > A lot of those products are then enshittified and badly managed because large organisation politics screws things up. That's n…

I actually agree with most of this. A few points: SpaceX built great rockets before it became large (though this is relative - a small rocket company is a large hairdresser, for example). There is a certain scale required for some types of business, agreed. But getting larger doesn't necessarily make them better. > That description fits small business owners much better IMO. In our times, at least in non-failed weste…

> Flat disagree with this. Small org CEOs are close to their customers and employees and if they behave like dicks then they get punished quickly. Obviously some still do, because people, but it's harder for a small company CEO to continue being a dick.

I'm thinking it might be both - depending on who the real customers are.

I've seen plenty of what I described in "boring" B2C like... grocery stores. But thinking about it, for a grocery store chain, customers are as much a commodity as the products they buy. Suppliers are where relationships (and power plays) matter.

(This might be fundamentally the same problem as the infamous case of "enterprise software procurement" - people using the software aren't the ones paying for it. For a grocery store chain, customers come and go all the time for many reasons, so it averages out anyway - but your suppliers and partners are what makes a difference in your bottom line.)

> And all commercial airplanes, MRIs, anything, were built first by small organisations, and only later by large orgs. Large orgs just can't invent new things unless they form specialist small orgs to do it (skunkworks, or Palo Alto, or similar). Large orgs just don't work like that.

Which is why I tried to point out the category error. "Large org with skunkworks" vs. "Bunch of smaller orgs forming an alliance and acquiring more smaller orgs to productionize a new technology" vs. "government megaproject" - they're all similar, arguably for a given invention they may very well be the same thing. Names and legal groupings are different, but the dynamics is (by anthropic principle) specific to what's needed for a given type of invention.

E.g. for stuff like airplanes or MRIs, you need individuals and small teams with lots of freedom (and a "hold my beer and watch this" culture often helps), but that gives you a prototype at best - scaling this so it works reliably, and then optimizing so it can be economical, both require throwing money at people doing boring work that mostly increments things on margin. And then the money has to come from someone, and someone must be willing to spend it to fund it all.

The actual org charts and legal charters don't matter - what matters is the incentives inside. I somewhat tentatively put forth a hypothesis: large orgs form to solve problems that the regular free market dynamics can't handle, by creating areas governed by different rule sets, within which that work can be done. Whether that's by fiat or corporate charter or a bunch of friends aligning their small businesses for the same goal, is window dressing.

Re: Claude Code is steganographically marking requests

#789

Earlier quoted context omitted.

So block people, instead of having false positives be secretly fucked over, and having them pay for the pleasure? Given the hidden model degradation of fable and now this, what makes you think this is where it stops? That's just what we know about and there's clearly a long-standing and deeply rooted malicious intent here. I've had Claude fuck over clean well documented code-bases for no reason, and there's a good ch…

So don’t use it!

It's like "if you don't like it here just move". Not really good advice, doesn't fix anything at all.

Re: Claude Code is steganographically marking requests

#790
post #785

Earlier quoted context omitted.

Ah yes. The exceptionalism argument. There's the good "us" and the bad "them."

I didn't make an exceptionalist argument, but if any country's behavior and values can be measured compared to others, you will always be able to make some kind of decision about where those fall in terms of goodness or badness. Do you not believe in good and bad?

I believe in good and bad.

I don't believe in US good, non-US bad. I also don't believe the same about my religion or my political party, for that matter.

How you measure depends on weights you assign (cultural system of values) and what information you use (media bias).

You can rank in the extremes (e.g. North Korea as worse than Belgium), since they come out that way by almost any set of information and values. Comparing the US to most other countries, there isn't a clear ordering. If you believe the things you wrote, I think the other comment summed it up well: "You sound like you've swallowed pro-US-propaganda hook, line and sinker."

Most countries have similar propaganda, by the way.

Post reply on HN