Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The text in Claude Code’s “Extended Thinking” output

patrickmccanna.net

Post reply on HN
Print view View original

121–130 of 248 posts

Re: The text in Claude Code’s “Extended Thinking” output

#121

Post by anuramat »

dragonwriter wrote: post #63
anuramat wrote: post #16

Earlier quoted context omitted.

how is summarized CoT a moat, and how is having the top 2 LLMs not a moat?

Not revealing actual thinking traces prevents mdoel distillation on yhe actual output (thinking traces are a key part of the output) which makes it harder for conpetitors to catch up (a moat). Being currently in the lead in a category is not a moat,a moat is whatever creates a barrier to competitors catching up when you are in the lead. Merely being in the lead is not a moat except in a market with strong network ext…

unrestricted access to better models at compute prices = better synthetic data and faster research, so its not just about the product imho

Re: The text in Claude Code’s “Extended Thinking” output

#122

Post by red75prime »

_aavaa_ wrote: post #37
StizzurpXDD wrote: post #18

This is not just Anthropic. Almost all big AI companies, including OpenAI and Google, hide their model's actual reasoning. This is because revealing the raw reasoning exposes exactly how the AI processes information. These companies spend in huge amounts on R&D to develop a thinking process that is superior to their competition. Exposing those thinking mechanics to competitors would completely defeat the purpose of t…

Or like providing the world’s information in machine readable format that the AI companies can convert into model weights without getting permission or compensating the rights holders

"Your text batch moved the weights away from the final values. Your contribution is negative."

Re: The text in Claude Code’s “Extended Thinking” output

#123

Post by segmondy »

What I find sad is how much Anthropic goes to hide your data, yet they are happy to slurp up all yours and most of you are happy to hand it over. ... then they turn around and compete with you by building your products that eat into your market. Anthropic believes their reasoning tokens is a moat and that it's giving other labs an edge and that's why they are hiding it. If they really believe that is their edge, then they are in for a surprise.

Re: The text in Claude Code’s “Extended Thinking” output

#124

Post by lsdmtme »

linsomniac wrote: post #95

I feel like I get a lot of what this article presents as "hidden" by using this process: - "Read `description` and create a specification, implementation guide, and checklist." - "Ask clarifying questions. If any of those questions has a clear best recommendation, please select that yourself and record that in "autorecommendations.md". - "Have codex and antigravity review each of these and work to consensus." These a…

What are you using exactly to have claude code natively interact with codex and antigravity?

Mind sharing your prompts?

Re: The text in Claude Code’s “Extended Thinking” output

#125

Post by ForHackernews »

Whatever it says is not always what it is doing https://transformer-circuits.pub/2025/attribution-graphs/bio...

> The computation we can see looks like it’s just guessing the answer, despite the chain of thought suggesting it’s computed it using a calculator.

It might be hallucinating or lying, it's not like you are actually observing the internals of the model.

Re: The text in Claude Code’s “Extended Thinking” output

#126

Post by Silagi »

glaslong wrote: post #35
altmanaltman wrote: post #27

Earlier quoted context omitted.

also a typo in the last sentence you're vrs your

Weirdly pleasant, if minor, signal of human authorship

I'm convinced this "signal" has already been hijacked. Maybe a Baader-Meinhof phenomenon, but I've noticed more and more egregious spelling errors that make little sense from a human perspective. Hop into whatever chatbot you'd like and ask it to "write a paragraph with subtle misspellings on long but common words", and you'll notice misspellings that just feel wrong, because they don't map to a clear misunderstanding that a person could have.

Or maybe I'm losing it after reading too much slop. Also distinctly possible.

Re: The text in Claude Code’s “Extended Thinking” output

#128

Post by JamesSwift »

Roritharr wrote: post #85
JamesSwift wrote: post #75

Earlier quoted context omitted.

> all major vendors throw out the reasoning tokens between turns That would be surprising to me. The reasoning _is_ the model intelligence in a lot of respects, and so dropping those from the context would affect its output pretty significantly. I assume that instead they just have a lot of guardrails in place and multiple runtime environments that an individual turns ping-pong between in order to dehydrate/rehydrate…

Anthropic very explicitly says below their diagrams ( https://platform.claude.com/docs/en/build-with-claude/contex... ) on this: "Stripping extended thinking: Extended thinking blocks (shown in dark gray) are generated during each turn's output phase, but are not carried forward as input tokens for subsequent turns. You do not need to strip the thinking blocks yourself. The Claude API automatically does this for you…

Thats really surprising, I stand corrected. I have had a lot of issues with hallucinations I attributed to adaptive thinking, but I wonder if those were actually due to this behavior instead.

I also wonder if they actually do a hybrid of "standard reasoning" and then classify this stripped chain of thought as "extended thinking".

Re: The text in Claude Code’s “Extended Thinking” output

#129

Post by kccqzy »

VulgarExigency wrote: post #109
datastoat wrote: post #89

I believe that chain-of-thought reasoning blocks don't really correspond to what humans think of as reasoning. (See section 6.2.2 of the Fable/Mythos system card about "illegible reasoning", and the questions raised by the Apple paper on "The illusion of thinking".) I assumed they obscure the reasoning blocks because if users saw what's going on they'd be alarmed. Just as I'd probably be alarmed if I saw what was rea…

I've said "what the FUCK are you THINKING" more times than I can count when reading Deepseek or GLM chains-of-thought only for them to end at the correct answer. Other times, they have useful ideas there that they leave out of their answers.

Yeah when I read a model’s chains-of-thought I have a tendency to interrupt that because it’s going down a wrong direction. But usually the end result is still fine.

Re: The text in Claude Code’s “Extended Thinking” output

#130

Post by CamperBob2 »

VulgarExigency wrote: post #109
datastoat wrote: post #89

I believe that chain-of-thought reasoning blocks don't really correspond to what humans think of as reasoning. (See section 6.2.2 of the Fable/Mythos system card about "illegible reasoning", and the questions raised by the Apple paper on "The illusion of thinking".) I assumed they obscure the reasoning blocks because if users saw what's going on they'd be alarmed. Just as I'd probably be alarmed if I saw what was rea…

I've said "what the FUCK are you THINKING" more times than I can count when reading Deepseek or GLM chains-of-thought only for them to end at the correct answer. Other times, they have useful ideas there that they leave out of their answers.

It's similar to the process that transformers use when you ask them to do arithmetic without tools, I think. Some CoT tokens must be emitted up front for use as a computational substrate, but exactly what tokens they are isn't necessarily important or relevant to the final answer. And when that answer is returned, it may not be possible to tell what the actual reasoning process looked like behind the scenes.

It only makes sense that the same mechanism comes into play in strictly-verbal contexts.

Also, this is why "distillation attacks" are largely bullshit that Anthropic spreads for political purposes. Proper distillation requires access to the logits.

Post reply on HN