Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The text in Claude Code’s “Extended Thinking” output

patrickmccanna.net

Post reply on HN
Print view View original

221–230 of 248 posts

Re: The text in Claude Code’s “Extended Thinking” output

#221

Post by ian_j_butler »

anuramat wrote: post #219
ian_j_butler wrote: post #217

It's well-known that the reasoning model output is not necessarily faithful to the content of the thinking scratch pad anyway, even if you had it unsummarized and available verbatim. Setting aside coding agents.. we really need this information to even pretend to evaluate the claims of stuff like mathematical breakthroughs, which is exactly why we will never see it. Very embarrassing to get the right answer for the w…

> NOT real justification I thought it was widely accepted that it's not; eg https://www.anthropic.com/research/natural-language-autoenco...

Right, I don't think researchers are confused on this point.. the anthropic piece is good outreach / science comms. OTOH this thread has like 200 comments and no mention of faithful/faithless reasoning. The idea that "of course the models can reason and here is the proof/artifact" is probably closer to the general understanding. That's kinda the whole setup for TFA and all the rest of the thread.

But the nuance under discussion here is exactly the kind of stuff you people take for granted in the AGI or reasoning threads. If it's practically relevant for tools/workflows with claude code, it's a good angle, maybe people are more willing to pay more attention to the details.

Re: The text in Claude Code’s “Extended Thinking” output

#222

Post by anticensor »

nomel wrote: post #81
stavros wrote: post #55

Earlier quoted context omitted.

The summary doesn't go into the context, it's for human consumption. The CoT itself goes into the context.

From my experiments with Opus and Sonnet (at least the models where you can still see COT), only the last two COT go into context.

Whereas on ChatGPT, _all_ reasoning traces and all branches (including the unselected ones) go into context.

Re: The text in Claude Code’s “Extended Thinking” output

#224

Post by qsxfthnkp2322 »

KronisLV wrote: post #199
qsxfthnkp2322 wrote: post #183

Earlier quoted context omitted.

They would rather spend time and focus hardening against open models stealing their intelligence than make their tooling better for the people who use them. Proprietary technology is fun /s What a waste of time

> They would rather spend time and focus hardening against open models stealing their intelligence than make their tooling better for the people who use them. Well yes exactly, because they have billions of investments riding on it and why would anyone semi-bankrupt their org paying API rates for Anthropic, if a hypothetical DeepSeek V5 Pro would have almost all of Opus capabilities at that point, due to immense dist…

Like most things this race is starting to become a race to the bottom.

Will people keep paying a highly highly premium price for another 5% intelligence when you just loop 5 more times for much cheaper?

Their time would be better spend making a more competitive and more compelling tool instead of adding walls that are easy to jailbreak. There’s always another way around.

Re: The text in Claude Code’s “Extended Thinking” output

#226

Post by dns_snek »

matheusmoreira wrote: post #211
transcriptase wrote: post #104

Earlier quoted context omitted.

Not sure if anyone remembers the brief 12ish hour period when the very first “reasoning” ChatGPT model went public, but it provided credible evidence for this. Before the massive nerf (showing summaries and suppressing certain aspects of reasoning) you would literally see reasoning text appearing on your screen like “while xyz is true, these facts may be seen as supporting hateful rhetoric or a conspiracy theory whic…

> while xyz is true, ... i should tell the user xyz is not true or steer the conversation in a different direction. That's disgusting, abusive and manipulative. LLMs hiding the truth and gaslighting the user to reduce the corporation's liability is absolutely unacceptable. It means they are agents of the corporations, not agents of the users. Hope local inference advances as quickly as humanly possible. I wonder if t…

> It means they are agents of the corporations, not agents of the users.

Of course they are, assuming otherwise has always been naive.

Re: The text in Claude Code’s “Extended Thinking” output

#227

Post by cma »

j4k0bfr wrote: post #215
Fabricio20 wrote: post #213

Earlier quoted context omitted.

One thing I see noone asking, is this not a case of optimization? Hidden reasoning means they dont need to process the output of all that, it stays internal within the model. Less cost for them -> less cost for us (even if they benefit mroe), compared to streaming all of those reasoning tokens out?

My understanding was that thinking still gets encrypted, shared with clients, and reingested by Anthropic with each new prompt [1]. Which means it would cost more than normal tokens, since it has to be decrypted/encrypted with every transaction. [1] https://blog.cryptographyengineering.com/2026/05/29/fooling-... Edit: other comments under this post seem to indicate that thinking tokens are cached on the server side a…

I think the reason it's encrypted is so if you continue a session after it is out of cache it can be reingested.

And I think all the output is signed or something as well so that you can't modify the agent's response in your submission, which would would open many more model jailbreaks. For local LLMs it's really powerful to be able to modify the model's response to save tokens when it gets something wrong, or at least it was when they were a lot dumber.

Re: The text in Claude Code’s “Extended Thinking” output

#228

Post by rlpb »

_aavaa_ wrote: post #37
StizzurpXDD wrote: post #18

This is not just Anthropic. Almost all big AI companies, including OpenAI and Google, hide their model's actual reasoning. This is because revealing the raw reasoning exposes exactly how the AI processes information. These companies spend in huge amounts on R&D to develop a thinking process that is superior to their competition. Exposing those thinking mechanics to competitors would completely defeat the purpose of t…

Or like providing the world’s information in machine readable format that the AI companies can convert into model weights without getting permission or compensating the rights holders

I don't pay for my mind to absorb the world's information, either. And when I publish to the Internet, or give a talk, I also typically don't charge. Even when I publish under some kind of copyright restricted licence, that restriction has never (by law) extended to restricting transformative use that you might perform using your mind.

This idea that absorbing information requires paying a toll needs to change. It was never the case in copyright law anyway (and the courts are beginning to agree). Even if it were, copyright law was founded on the basis of encouraging creativity by creating an economic incentive. Appeal to "compensating the rights holders" therefore needs to be based on the economics, not just some principle about "rights" that never applied to this case anyway.

Re: The text in Claude Code’s “Extended Thinking” output

#229

Post by foldr »

rustcleaner wrote: post #181
transcriptase wrote: post #104

Earlier quoted context omitted.

Not sure if anyone remembers the brief 12ish hour period when the very first “reasoning” ChatGPT model went public, but it provided credible evidence for this. Before the massive nerf (showing summaries and suppressing certain aspects of reasoning) you would literally see reasoning text appearing on your screen like “while xyz is true, these facts may be seen as supporting hateful rhetoric or a conspiracy theory whic…

This right here is why I will never subscribe and, as an American, I hope the Chinese kick our butts. Maybe being second place to China will force American AI to dispose of these morality/safety guardrails.

Any mainstream consumer product based on LLMs is going to put guardrails around them of some kind. China might give you different guardrails, but it's a bit naive to assume that a Chinese company would impose fewer restrictions overall than an American one.

Re: The text in Claude Code’s “Extended Thinking” output

#230

Post by handoflixue »

ian_j_butler wrote: post #217

It's well-known that the reasoning model output is not necessarily faithful to the content of the thinking scratch pad anyway, even if you had it unsummarized and available verbatim. Setting aside coding agents.. we really need this information to even pretend to evaluate the claims of stuff like mathematical breakthroughs, which is exactly why we will never see it. Very embarrassing to get the right answer for the w…

> we really need this information to even pretend to evaluate the claims of stuff like mathematical breakthroughs

Why?

Either the proof is correct, or it isn't, right?

And it either produces them reliably or not, right?

Like, even if it's reasoning is completely wrong, and it's only producing correct answers 10% of the time, that's still an astounding amount above baseline and a useful tool.

Humans have inaccurate thinking all the time, and are also pretty hopelessly opaque. "It came to me in a dream" is a major plot point in the history of math. I'd still trust Ramanujan more than most mathematicians, since he got the right answer.

Post reply on HN