Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The text in Claude Code’s “Extended Thinking” output

patrickmccanna.net

Post reply on HN
Print view View original

141–150 of 248 posts

Re: The text in Claude Code’s “Extended Thinking” output

#141

Post by tsunamifury »

orangecat wrote: post #65
tsunamifury wrote: post #7

It’s not surprising than the Sota model makers core goal is to get user dependent while denying them increasing amounts of understanding of how it works to form a deeply unhealthy dependency. Tell me this. If you hired a junior engineer or designer who refused to explain their thinking on their code and how they solved for the spec what would you do? (That being said the reasoning output is still a summary of the Kvc…

* If you hired a junior engineer or designer who refused to explain their thinking on their code* Any explanation that someone gives of their thinking process is necessarily lossy and likely partially confabulated.

Did you not even bother to read to even the end of the comment before jumping at 'correcting' someone?

Re: The text in Claude Code’s “Extended Thinking” output

#142

Post by sigmoid10 »

tough wrote: post #120
Roritharr wrote: post #70

Earlier quoted context omitted.

I've thought about the high-jacking of reasoning-chains as a potential vector, but never saw a proven implementation in american models since, from my understanding, all major vendors throw out the reasoning tokens between turns.

OAI is now implementing encrypted CoT that you can store and pass back between turns (harness call), so new models have it https://developers.openai.com/api/docs/guides/reasoning#encr...

You could also use the responses api which stores all message contents (including reasoning) on OAI servers. This has been possible for quite a while now. Encryption is only necessary if you really care about local storage (which is different from privacy concerns, because the data gets sent to their servers anyway).

Re: The text in Claude Code’s “Extended Thinking” output

#143

Post by 8note »

btown wrote: post #86
Roritharr wrote: post #70

Earlier quoted context omitted.

I've thought about the high-jacking of reasoning-chains as a potential vector, but never saw a proven implementation in american models since, from my understanding, all major vendors throw out the reasoning tokens between turns.

For Claude, at least, "throw out the reasoning tokens" is only true when a session has been idle for more than an hour, and is new since March. The basic concept is that for a session active recently, interleaved thinking tokens are already in KV cache, so it's more efficient to keep using them than not! But when resuming an older session where KV cache has been evicted, it's more expensive to restore the thinking to…

its mostly annoying in that you give opus a big job, that should be able to run for hours on end, but instead it tries to stop and checkpoint at every soonest possible moment even though the rest of the work is well specced and ready to go.

then it waits for the hour and gets dumbed down

Re: The text in Claude Code’s “Extended Thinking” output

#144

Post by paweladamczuk »

irthomasthomas wrote: post #51

I won't use or recommend models with hidden reasoning, (thats all American models). It's too much of a risk and makes prompt optimization harder. Risky because it makes it possible for an attacker to prompt inject the reasoning chain to carry out a secret objective, and to hide that from the summary and output. Interleaved reasoning and function calling makes this even more dangerous. A model can call functions durin…

I don't think there can be tool calls inside the obfuscated reasoning blocks. I mean, in order for those function calls to be evaluated client-side, that thinking stream would have to be decrypted on the client side at some point, which would defeat the purpose of obfuscating it the way they do.

If you mean the function calls might happen server side, there is nothing preventing the server from doing it and hiding it from you as long as you are using an API for inference.

Re: The text in Claude Code’s “Extended Thinking” output

#145

Post by panikal »

mannanj wrote: post #135
segmondy wrote: post #123

What I find sad is how much Anthropic goes to hide your data, yet they are happy to slurp up all yours and most of you are happy to hand it over. ... then they turn around and compete with you by building your products that eat into your market. Anthropic believes their reasoning tokens is a moat and that it's giving other labs an edge and that's why they are hiding it. If they really believe that is their edge, then…

I don't think people are happy to give it over, gullible and naive maybe?

[dead]

Re: The text in Claude Code’s “Extended Thinking” output

#146

Post by Bolwin »

irthomasthomas wrote: post #51

I won't use or recommend models with hidden reasoning, (thats all American models). It's too much of a risk and makes prompt optimization harder. Risky because it makes it possible for an attacker to prompt inject the reasoning chain to carry out a secret objective, and to hide that from the summary and output. Interleaved reasoning and function calling makes this even more dangerous. A model can call functions durin…

> Interleaved reasoning and function calling makes this even more dangerous. A model can call functions during the hidden reasoning phase.

The reasoning may be hidden but the tool calls are not, how else would the client execute them

Re: The text in Claude Code’s “Extended Thinking” output

#148

Post by pixlmint »

irthomasthomas wrote: post #51

I won't use or recommend models with hidden reasoning, (thats all American models). It's too much of a risk and makes prompt optimization harder. Risky because it makes it possible for an attacker to prompt inject the reasoning chain to carry out a secret objective, and to hide that from the summary and output. Interleaved reasoning and function calling makes this even more dangerous. A model can call functions durin…

Do they do the same when using the model through API in something like Opencode?

Re: The text in Claude Code’s “Extended Thinking” output

#149

Post by wren6991 »

CamperBob2 wrote: post #130
VulgarExigency wrote: post #109

Earlier quoted context omitted.

I've said "what the FUCK are you THINKING" more times than I can count when reading Deepseek or GLM chains-of-thought only for them to end at the correct answer. Other times, they have useful ideas there that they leave out of their answers.

It's similar to the process that transformers use when you ask them to do arithmetic without tools, I think. Some CoT tokens must be emitted up front for use as a computational substrate, but exactly what tokens they are isn't necessarily important or relevant to the final answer. And when that answer is returned, it may not be possible to tell what the actual reasoning process looked like behind the scenes. It only…

> Proper distillation requires access to the logits

Why do you need logits? Can't you just train on cross-entropy loss of the model against the hard decision, like you do in regular pretraining?

There are definitely current-gen open-weight models (Step 3.7 Flash is one) that refer to themselves as an OpenAI model in CoT, but not in the final response.

Re: The text in Claude Code’s “Extended Thinking” output

#150

Post by JamesSwift »

haus20xx wrote: post #136
Roritharr wrote: post #85

Earlier quoted context omitted.

Anthropic very explicitly says below their diagrams ( https://platform.claude.com/docs/en/build-with-claude/contex... ) on this: "Stripping extended thinking: Extended thinking blocks (shown in dark gray) are generated during each turn's output phase, but are not carried forward as input tokens for subsequent turns. You do not need to strip the thinking blocks yourself. The Claude API automatically does this for you…

https://platform.claude.com/docs/en/build-with-claude/extend... default depends on the model class. Opus: Claude Opus 4.5 and later Opus models keep all prior thinking blocks; Claude Opus 4.1 (deprecated) and earlier Opus models keep only the last assistant turn's thinking. Sonnet: Claude Sonnet 4.6 and later Sonnet models keep all; Claude Sonnet 4.5 and earlier Sonnet models keep only the last turn. Haiku: all Haiku…

Now Im even more confused : D

That would also explain the issue I mention in my other comment. And would also reinforce how much output would degrade without this. Opus 4.5 was a step above previous models in my experience. At some point it degraded and only got better when I disabled adaptive thinking. Adaptive thinking is always on for 4.6 and above.

Post reply on HN