Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Stealing Reasoning Traces from Proprietary LLM APIs

stolen-thoughts.com

Post reply on HN
Print view View original

171–180 of 325 posts

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#171

Post by TeMPOraL »

QuercusMax wrote: post #95
SaltyBackendGuy wrote: post #82

Earlier quoted context omitted.

Or even my fable credits run out mid task and need to switch back to opus >.<

Prior to LLMs I never considered that I might have to make a resource-usage decision between hiring Star Trek's Data vs. his stupider brother B4... https://memory-alpha.fandom.com/wiki/B-4

Star Trek is a post-scarcity society, those problems don't exist there unless you're in the middle of a crisis and on emergency power.

LLMs briefly seemed like this too, after subscriptions made the SOTA models too cheap to meter, but before they walked back on that and introduced quotas...

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#172

Post by polymer8563 »

HarHarVeryFunny wrote: post #86
dgellow wrote: post #85

Earlier quoted context omitted.

Correct, yes. It’s delightedly simple. And they validate by asserting the reasoning token length matches

For all of the years of research, thinking and talking about model alignment, safety, confinement, etc, when it comes down to it these companies appear to be entirely incompetent.

for safety in particular it's pure theater, they only care as long as the orange guy thinks it's safe from "enemies of freedom"

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#173

Post by polymer8563 »

azinman2 wrote: post #19
myworkaccount2 wrote: post #17

Is this how the eastern labs "distill" SOTA models? If you can play it right, you don't even need to send suspicious prompts to the frontier models. Just use them for regular tasks, extract the encrypted COT blocks and replay it to a cheaper model to get the plain text COT. But the real question is: Is it okay to steal from a thief's hoard?

The reasoning blocks are not stolen/mined from the internet at large directly. They’re the result of a lot of research, time, money, and expertise into creating a reasoning model. To me the answer is quite clearly no, especially when the encrypted blocks demonstrate they want to protect it.

if you make reasoning soup of my data withou my consent it still is my data and i did not ask for your reasoning soup

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#174

Post by infecto »

paxys wrote: post #152
infecto wrote: post #144

Wouldn’t the fix be to encrypt before the api call hits the LLM? You would encrypt/decrypt at a separate layer than the LLM. I am sure i am missing something but would love to be educated.

The LLM needs to read the CoT as part of the conversation. You can ask the models to share them with you. Stronger models will refuse, while weaker ones can be “jailbroken”.

I don’t think that answers what I am wondering. Asked differently why is encoding/decoding the cot the concern of the LLM?

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#175

Post by gruez »

breezybottom wrote: post #131
Perseids wrote: post #104

Earlier quoted context omitted.

I think they are doing us a disservice by perpetuating this focus on intellectual "property" [1] in regard to AI. The real issue with Anthropic, OpenAI etc. is not that they have used all of our public knowledge for training their LLMs. Creating new work from old and learning from prior generations is what we all do. The issue is that they want to claim all of the benefits for themselves. They are standing on the sho…

That is the issue, you just rephrased it to sound softer (public knowledge isn't a legal term). Fair use law isn't supposed to apply to commercial activity. So it is absolutely about using copyrighted work for LLM training.

>Fair use law isn't supposed to apply to commercial activity.

No, otherwise there would be a straightforward "non-commercial" clause. Instead there's a 4 part test, which takes usage (commercial or not) into account, but doesn't hinge solely on it.

https://en.wikipedia.org/wiki/Fair_use

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#176

Post by [deleted] »

anigbrowl wrote: post #164
throw1234567891 wrote: post #156

Earlier quoted context omitted.

No, you paid for the end result. The thought process is a step in between, a function. Think about it, who should get charged if the answer you received comes from a cache? Thinking tokens are the complexity-of-the-problem cost. I mean, you may not agree but both are valid points of view.

No I didn't. I buy my tokens from a provider that exposes the model reasoning so I can understand what it's doing and work with it, or interrupt if I see things going in the wrong direction.

[deleted]

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#177

Post by paxys »

infecto wrote: post #174
paxys wrote: post #152

Earlier quoted context omitted.

The LLM needs to read the CoT as part of the conversation. You can ask the models to share them with you. Stronger models will refuse, while weaker ones can be “jailbroken”.

I don’t think that answers what I am wondering. Asked differently why is encoding/decoding the cot the concern of the LLM?

It isn’t the concern of the LLM. Regardless of where the encryption/decryption is happening, the issue is that the LLM needs to access the raw CoT.

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#178

Post by infecto »

paxys wrote: post #177
infecto wrote: post #174

Earlier quoted context omitted.

I don’t think that answers what I am wondering. Asked differently why is encoding/decoding the cot the concern of the LLM?

It isn’t the concern of the LLM. Regardless of where the encryption/decryption is happening, the issue is that the LLM needs to access the raw CoT.

Again I don’t think you’re really getting at what I am asking. Sorry. My whole point was why does the LLM have access of decrypting. It should happen outside of the LLM layer.

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#179

Post by yubblegum »

paxys wrote: post #154
yubblegum wrote: post #105

Earlier quoted context omitted.

Sure, but if each session has a unique key then these need to be managed and stored and unauthorized access to these leaves tracks. So all that had to be 'compromised' is a single universally applicable key. Again, the question stands: session based encryption can be scalable and efficient. Why aren't they using it?

The exploit here isn’t a leaked encryption key. It’s pretty likely that they are already using a unique key per conversation. The raw CoT eventually reaches the model, and you can convince the model to share it with you.

If a different model is using encyrpted blocks of another model, then by definition it is no longer a session scoped bit of information. Since you can give it to any other session and another model, clearly it doesn't even have to be the same user. Therefore, there is only one (set) of universally available key(s) used by all models across all sessions.

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#180

Post by nonethewiser »

Aissen wrote: post #138

"Stealing" something you already paid for (tokens), but that you can't have access to(!). And trained on the sum of human knowledge. Training on other model outputs ought to be business as usual, stop using morally charged terms made up by future monopolists: https://thomasdullien.github.io/posts/2026-06-15-rl-economic...

> stop using morally charged terms made up by future monopolists

Lets not gloss over this claim. Being: “Stealing is a morally charged term made up by future monopolists.”

I strongly disagree. Stealing is not a made up term and property rights are foundational for any society. Your take is at least sensationalist if not malicious.

Post reply on HN