Stealing Reasoning Traces from Proprietary LLM APIs

Post by mlazos » Wed, Aug 12, 2026, 4:52 AM UTC

ashdksnndck wrote: post #224

Aissen wrote: post #138

"Stealing" something you already paid for (tokens), but that you can't have access to(!). And trained on the sum of human knowledge. Training on other model outputs ought to be business as usual, stop using morally charged terms made up by future monopolists: https://thomasdullien.github.io/posts/2026-06-15-rl-economic...

Suppose you hired a consulting firm to write a report, and they delivered the report but not the internal conversations they had when developing it. You exploit a vulnerability in their phone system to get those conversations. You can argue over semantics of whether “theft” is what you did, maybe the right word is “espionage” or “spying”, but that either way we probably agree you are guilty of something? Paying for t…

Post by blackqueeriroh » Wed, Aug 12, 2026, 5:02 AM UTC

matheusmoreira wrote: post #205

nonethewiser wrote: post #196

Earlier quoted context omitted.

Reread what he called a morally charged, made up term by future monopolists. Stealing. Not distilling, not stealing “non rivalrous goods.” Just stealing. Do you agree with what he actually said?

Yes, "tokens, trained on the sum of human knowledge", are the most absolutely, unambiguously clear example of a non-rivalrous good I've ever seen. The current copyright status quo has them placed in the public domain . There is literally nothing wrong with "stealing" those tokens. They have exactly zero legal protection. "Stealing" those AI output tokens is so fundamentally impossible that it wouldn't be "stealing" i…

Post by glub » Wed, Aug 12, 2026, 5:07 AM UTC

varenc wrote: post #226

glub wrote: post #213

Earlier quoted context omitted.

The only "secret" there is a very basic instruction that the model receives, like "summarize current state and upcoming work" before compaction - same model that was just running your inference, with same cache, only server side, with no extra tools or capabilities. Then the fresh context gets the output from that as an encrypted blob + codex then injects up to 64k tokens of previous conversation, the latter part is…

The compaction prompt doesn't seem like the valuable thing here. I suspect they're protecting the compaction result itself. If you're trying to distill a model, collecting lots of examples on how a large conversation gets compacted to a smaller summary is particularly useful data.

Post by blackqueeriroh » Wed, Aug 12, 2026, 5:11 AM UTC

cyanydeez wrote: post #68

niemandhier wrote: post #59

You cannot steal what is not owned. At least in the EU there is no copyright for LLM outputs, so I guess all they might do is violate the terms of service.

they also can't claim to be the owner of the output either because they can't claim to have had a valid license for the inputs.

Post by MaxMatti » Wed, Aug 12, 2026, 5:12 AM UTC

ashirviskas wrote: post #222

Pragmata wrote: post #119

Apparently you can do the same by simply running it without reasoning, while giving it a thinking tool... >guys you do know you can just disable thinking, and instead give it a "deep_think" tool, and it will call it with internal CoT reasoning format right? >gl fixing that https://x.com/_can1357/status/2087228354399265125?s=20

I've been doing that since before reasoning was a thing baked into the models, it always performs better this way. Except for some providers/models where you just can't easily turn it off, now I just avoid them. This way I save tokens and have full control of the reasoning.