Live data from Hacker News

Stealing Reasoning Traces from Proprietary LLM APIs

stolen-thoughts.com

151–160 of 325 posts

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#151
post #138

"Stealing" something you already paid for (tokens), but that you can't have access to(!). And trained on the sum of human knowledge. Training on other model outputs ought to be business as usual, stop using morally charged terms made up by future monopolists: https://thomasdullien.github.io/posts/2026-06-15-rl-economic...

Liberating!

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#152

Wouldn’t the fix be to encrypt before the api call hits the LLM? You would encrypt/decrypt at a separate layer than the LLM. I am sure i am missing something but would love to be educated.

The LLM needs to read the CoT as part of the conversation. You can ask the models to share them with you. Stronger models will refuse, while weaker ones can be “jailbroken”.

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#153
post #138

"Stealing" something you already paid for (tokens), but that you can't have access to(!). And trained on the sum of human knowledge. Training on other model outputs ought to be business as usual, stop using morally charged terms made up by future monopolists: https://thomasdullien.github.io/posts/2026-06-15-rl-economic...

The only person calling it stealing is the author of this article, so this is a pointless discussion. The majority of this thread is just arguing with themselves.

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#154
post #83

Earlier quoted context omitted.

The provider has the hidden text anyway; this isn’t customer managed encryption.

Sure, but if each session has a unique key then these need to be managed and stored and unauthorized access to these leaves tracks. So all that had to be 'compromised' is a single universally applicable key. Again, the question stands: session based encryption can be scalable and efficient. Why aren't they using it?

The exploit here isn’t a leaked encryption key. It’s pretty likely that they are already using a unique key per conversation. The raw CoT eventually reaches the model, and you can convince the model to share it with you.

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#155

Earlier quoted context omitted.

I think they are doing us a disservice by perpetuating this focus on intellectual "property" [1] in regard to AI. The real issue with Anthropic, OpenAI etc. is not that they have used all of our public knowledge for training their LLMs. Creating new work from old and learning from prior generations is what we all do. The issue is that they want to claim all of the benefits for themselves. They are standing on the sho…

That is the issue, you just rephrased it to sound softer (public knowledge isn't a legal term). Fair use law isn't supposed to apply to commercial activity. So it is absolutely about using copyrighted work for LLM training.

The thing is, I explicitly wanted to not focus on current legality, because I believe this is leading us astray. This is about how we want to shape our society, not about how their actions fit into a legal system that wasn't designed to handle AI training. If, by chance, all of this was legal, the same arguments would still apply. For example, see how Adobe is training their image generating AI on licensed work. The result -- privatized intelligence -- is the same as when illegitimate training data would have been used. Or, say, an AI company makes a deal with all the publishers to license their books for training. Or a court decides that training AI with MIT licensed code is totally fine. Or an AI company pays legions of software developers to train their AI. All of this still ends in a power grab. All of this still privatizes knowledge and skills accumulated over centuries, if not millennia.

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#156
post #138

"Stealing" something you already paid for (tokens), but that you can't have access to(!). And trained on the sum of human knowledge. Training on other model outputs ought to be business as usual, stop using morally charged terms made up by future monopolists: https://thomasdullien.github.io/posts/2026-06-15-rl-economic...

No, you paid for the end result. The thought process is a step in between, a function. Think about it, who should get charged if the answer you received comes from a cache? Thinking tokens are the complexity-of-the-problem cost. I mean, you may not agree but both are valid points of view.

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#157

Is this how the eastern labs "distill" SOTA models? If you can play it right, you don't even need to send suspicious prompts to the frontier models. Just use them for regular tasks, extract the encrypted COT blocks and replay it to a cheaper model to get the plain text COT. But the real question is: Is it okay to steal from a thief's hoard?

> But the real question is: Is it okay to steal By definition it cannot be stealing since you're paying for the tokens. It may be against their ToS, depending on what you end up doing with those tokens, but it cannot be stealing. If they charge by the token, all your tokens are belong to you :) I also find it very strange that everyone sort of accepts their ToS like no big deal. Imagine MS using the same terms for th…

I hesitate to nitpick with regard to something legal, given your username, but what makes this different from hiring a consultant with the agreement that their final output belongs to you, but you don't get access to their internal processes, tooling, notes, etc? Or a photographer where you get final edited prints, but you don't get the raw photos?

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#159

You cannot steal what is not owned. At least in the EU there is no copyright for LLM outputs, so I guess all they might do is violate the terms of service.

If I hire you for a consulting project, do I own the screen recording of your computer while you completed it?

Re: Stealing Reasoning Traces from Proprietary LLM APIs

#160

Apparently you can do the same by simply running it without reasoning, while giving it a thinking tool... >guys you do know you can just disable thinking, and instead give it a "deep_think" tool, and it will call it with internal CoT reasoning format right? >gl fixing that https://x.com/_can1357/status/2087228354399265125?s=20

its not exactly the same... its tool use spec asking to put thinking in inputs fields... it is a good idea but its not same.
Post reply on HN