Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The text in Claude Code’s “Extended Thinking” output

patrickmccanna.net

Post reply on HN
Print view View original

81–90 of 248 posts

Re: The text in Claude Code’s “Extended Thinking” output

#81

Post by nomel »

stavros wrote: post #55
_fat_santa wrote: post #13

IMHO I've never found the entire reasoning chain that particularly useful for my work. For me having a summary is honestly better from a context management perspective. I understand why they would encrypt it though, because those reasoning chains are VERY useful if you're distilling the model.

The summary doesn't go into the context, it's for human consumption. The CoT itself goes into the context.

From my experiments with Opus and Sonnet (at least the models where you can still see COT), only the last two COT go into context.

Re: The text in Claude Code’s “Extended Thinking” output

#82

Post by coldtea »

josefritzishere wrote: post #58

AI does not think. It is a word guessing machine. Anthropomorphizing technology does not add anything to our understanding.

A brain itself might be a guessing machine it's an established and actively studied research model of the human thought and the human brain.

Nor does knee jerk accusation of "anthropomorphizing" negate the fact that procedures that mimic human processing, even when done in software, are deservingly anthropomorphized, because they're a legitimate approximation of the human equivalent operations.

Re: The text in Claude Code’s “Extended Thinking” output

#83

Post by layer8 »

sigmar wrote: post #74

>the language in the docs is awfully indirect. writes this^ and then proceeds to highlight a bold title from the docs that says "summarized thinking" that explains things clearly in the first sentence. lol

The second sentence is making vague claims though.

Re: The text in Claude Code’s “Extended Thinking” output

#84

Post by irthomasthomas »

JamesSwift wrote: post #75
Roritharr wrote: post #70

Earlier quoted context omitted.

I've thought about the high-jacking of reasoning-chains as a potential vector, but never saw a proven implementation in american models since, from my understanding, all major vendors throw out the reasoning tokens between turns.

> all major vendors throw out the reasoning tokens between turns That would be surprising to me. The reasoning _is_ the model intelligence in a lot of respects, and so dropping those from the context would affect its output pretty significantly. I assume that instead they just have a lot of guardrails in place and multiple runtime environments that an individual turns ping-pong between in order to dehydrate/rehydrate…

Yep they store them encrypted https://blog.cryptographyengineering.com/2026/05/29/fooling-...

Re: The text in Claude Code’s “Extended Thinking” output

#85

Post by Roritharr »

JamesSwift wrote: post #75
Roritharr wrote: post #70

Earlier quoted context omitted.

I've thought about the high-jacking of reasoning-chains as a potential vector, but never saw a proven implementation in american models since, from my understanding, all major vendors throw out the reasoning tokens between turns.

> all major vendors throw out the reasoning tokens between turns That would be surprising to me. The reasoning _is_ the model intelligence in a lot of respects, and so dropping those from the context would affect its output pretty significantly. I assume that instead they just have a lot of guardrails in place and multiple runtime environments that an individual turns ping-pong between in order to dehydrate/rehydrate…

Anthropic very explicitly says below their diagrams ( https://platform.claude.com/docs/en/build-with-claude/contex... ) on this:

"Stripping extended thinking: Extended thinking blocks (shown in dark gray) are generated during each turn's output phase, but are not carried forward as input tokens for subsequent turns. You do not need to strip the thinking blocks yourself. The Claude API automatically does this for you if you pass them back."

It's more nuanced in the various modes, but i haven't seen it boil down towards Thinking Tokens surviving more than two turns.

Re: The text in Claude Code’s “Extended Thinking” output

#86

Post by btown »

Roritharr wrote: post #70
irthomasthomas wrote: post #51

I won't use or recommend models with hidden reasoning, (thats all American models). It's too much of a risk and makes prompt optimization harder. Risky because it makes it possible for an attacker to prompt inject the reasoning chain to carry out a secret objective, and to hide that from the summary and output. Interleaved reasoning and function calling makes this even more dangerous. A model can call functions durin…

I've thought about the high-jacking of reasoning-chains as a potential vector, but never saw a proven implementation in american models since, from my understanding, all major vendors throw out the reasoning tokens between turns.

For Claude, at least, "throw out the reasoning tokens" is only true when a session has been idle for more than an hour, and is new since March.

The basic concept is that for a session active recently, interleaved thinking tokens are already in KV cache, so it's more efficient to keep using them than not! But when resuming an older session where KV cache has been evicted, it's more expensive to restore the thinking tokens, so they're silently dropped from prior turns. It's 2026 and stateful servers are back on the menu!

https://www.anthropic.com/engineering/april-23-postmortem describes this as an intended optimization:

> The design should have been simple: if a session has been idle for more than an hour, we could reduce users’ cost of resuming that session by clearing old thinking sections. Since the request would be a cache miss anyway, we could prune unnecessary messages from the request to reduce the number of uncached tokens sent to the API. We’d then resume sending full reasoning history. To do this we used the clear_thinking_20251015 API header along with keep:1.

> The implementation had a bug. Instead of clearing thinking history once, it cleared it on every turn for the rest of the session... This surfaced as the forgetfulness, repetition, and odd tool choices people reported.

And https://news.ycombinator.com/item?id=47879561 is a thread with a Claude team member's further rationale.

> Eliding parts of the context after idle: old tool results, old messages, thinking. Of these, thinking performed the best, and when we shipped it, that's when we unintentionally introduced the bug in the blog post.

(Also, https://news.ycombinator.com/item?id=47884517 indicates OpenAI drops reasoning tokens "smartly" at its own election, which is likely a similar performance optimization.)

I've experimented with rules to have Claude Code be explicit about recapping its thinking tokens, including tool choices and approaches chosen and rejected, into actual message output, but this is lossy at best. And sometimes dropping reasoning tokens can give a session "fresh eyes" in a good way.

I just really don't like the lack of control, and it's a reminder of how ephemeral the current landscape is. The Claude giveth, and the Claude taketh away.

Re: The text in Claude Code’s “Extended Thinking” output

#87

Post by TeMPOraL »

reliablereason wrote: post #31

Is the thinking even done in real tokens? I thought it was done using the pure residual stream. That is instead of collapsing the residual stream to a token you treat the final layers output as a vector of size d_model and use that as input for the next position in the transformer. If that is the case thinking is not visible to us as users due to it not being done in text.

I saw that idea described as a step in AI 2027 (they call it "neuralese" and eyeballing the site, it's still labeled a hypothetical/future development), but AFAIK no one implemented/deployed this yet.

EDIT:

They link to a Meta paper from 2024/2025 though: https://arxiv.org/pdf/2412.06769/.

Re: The text in Claude Code’s “Extended Thinking” output

#88

Post by Roritharr »

btown wrote: post #86
Roritharr wrote: post #70

Earlier quoted context omitted.

I've thought about the high-jacking of reasoning-chains as a potential vector, but never saw a proven implementation in american models since, from my understanding, all major vendors throw out the reasoning tokens between turns.

For Claude, at least, "throw out the reasoning tokens" is only true when a session has been idle for more than an hour, and is new since March. The basic concept is that for a session active recently, interleaved thinking tokens are already in KV cache, so it's more efficient to keep using them than not! But when resuming an older session where KV cache has been evicted, it's more expensive to restore the thinking to…

Thank you! This is much more nuanced than my understanding so far!

Re: The text in Claude Code’s “Extended Thinking” output

#89

Post by datastoat »

I believe that chain-of-thought reasoning blocks don't really correspond to what humans think of as reasoning. (See section 6.2.2 of the Fable/Mythos system card about "illegible reasoning", and the questions raised by the Apple paper on "The illusion of thinking".) I assumed they obscure the reasoning blocks because if users saw what's going on they'd be alarmed. Just as I'd probably be alarmed if I saw what was really going on in the heads of my colleagues ...
Post reply on HN