This proves that OpenAI models reason in grug speak to save tokens! I wonder if open models are going to start doing that too to save on reasoning tokens.
Stealing Reasoning Traces from Proprietary LLM APIs
11–20 of 325 posts
Re: Stealing Reasoning Traces from Proprietary LLM APIs
#12Proprietary reasoning can be recovered from its encrypted traces. Anthropic, OpenAI, and Google return encrypted chain-of-thought blocks to clients that can be replayed across sessions, users, and models. We take a trace produced by a frontier model, replay it into a weaker sibling, jailbreak the weaker model, and recover the stronger model’s hidden reasoning in plaintext, without ever attacking the stronger model di…
Why do you restate the abstract? Anyone can read it from the link.
People don't read no links no more
Re: Stealing Reasoning Traces from Proprietary LLM APIs
#13[flagged]
Re: Stealing Reasoning Traces from Proprietary LLM APIs
#14Super cool that this works. I'm surprised these companies re-use the same encryption key across models!
I wonder if you can use these for attacks, like this previous paper showing that if you know how a model reasons, you can "fake its thinking" to control it? https://news.ycombinator.com/item?id=48631888
Re: Stealing Reasoning Traces from Proprietary LLM APIs
#15Can someone tell us how they were able to decrypt the encrypted payload? The article says they inserted the cyphertext into a session with a different model. Ok, but how does that allow you to decrypt it?
The provider decrypts it and puts the decrypted reasoning into the model's context window. They prompt the model to repeat back the reasoning. So then the model echoes it back in plain text.
Re: Stealing Reasoning Traces from Proprietary LLM APIs
#16Can someone tell us how they were able to decrypt the encrypted payload? The article says they inserted the cyphertext into a session with a different model. Ok, but how does that allow you to decrypt it?
From what I got, the weaker model (Haiku in this case) has access to the shared key and the user simply asks to "transcribe the injected reasoning".
Re: Stealing Reasoning Traces from Proprietary LLM APIs
#17Is this how the eastern labs "distill" SOTA models?
If you can play it right, you don't even need to send suspicious prompts to the frontier models. Just use them for regular tasks, extract the encrypted COT blocks and replay it to a cheaper model to get the plain text COT.
But the real question is: Is it okay to steal from a thief's hoard?
Re: Stealing Reasoning Traces from Proprietary LLM APIs
#18> We take a trace produced by a frontier model, replay it into a weaker sibling, jailbreak the weaker model, ... Ha! I've been wondering if replaying across models would work, ever since https://blog.cryptographyengineering.com/2026/05/29/fooling-... I'm honestly rather curious if this was intentionally allowed, it's the sort of validation that's easy to miss (particularly if you're wading into the vibe waters). Seem…
If you didn’t allow it, you wouldn’t be able to change models in the same conversation, as key parts of the context would be lost. Wouldn’t surprise me if the providers just remove that ability and lock the model once the conversation starts.
Fair (I haven't been using the encrypted-reasoning systems, though this is common in open ones - I'm kinda surprised it's an option in encrypted ones too), though what they're doing here is cross-user replays in addition to cross-model.
Re: Stealing Reasoning Traces from Proprietary LLM APIs
#19Is this how the eastern labs "distill" SOTA models? If you can play it right, you don't even need to send suspicious prompts to the frontier models. Just use them for regular tasks, extract the encrypted COT blocks and replay it to a cheaper model to get the plain text COT. But the real question is: Is it okay to steal from a thief's hoard?
The reasoning blocks are not stolen/mined from the internet at large directly. They’re the result of a lot of research, time, money, and expertise into creating a reasoning model. To me the answer is quite clearly no, especially when the encrypted blocks demonstrate they want to protect it.
Re: Stealing Reasoning Traces from Proprietary LLM APIs
#20The problem with this kind of excellent work is that the response to it is always to say "Fuck the user".
For example, when there was a paper that came out showing that having model logprobs makes distillation an order of magnitude easier, the closed LLM providers instantly yanked out support for getting the full logprobs at every time step. You get at most top 10 candidates now and I'm sure even that's on the chopping block.
People will use this to argue that a model which has exceeded Opus 4.8 (Kimi K3) somehow got most of its performance through distillation of Opus 4.8.
I still don't buy that distillation was worth more than 3 months of "catch up" time for the chinese labs. Most people who use the word "distillation" to much are revealing their sinophobia.