Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The text in Claude Code’s “Extended Thinking” output

patrickmccanna.net

Post reply on HN
Print view View original

61–70 of 248 posts

Re: The text in Claude Code’s “Extended Thinking” output

#61

Post by KellyCriterion »

stingraycharles wrote: post #34
craigmart wrote: post #24

This is something we have known for a very long time, and companies are not trying to hide that either. They do it to avoid letting competitors train their models on the CoTs

Yes hasn’t this been around since Opus 4.6? I very much recall this change happening around January or February, and it was very explicitly to prevent distillation. Sonnet does not have this limitation. Fun fact: if you go back to the old school from 2 years ago and provide explicit CoT prompts, you get the full thinking prompts back again! So you disable thinking altogether, and instead make thinking part of the reg…

- tada.wav -

Still, one of the daily most played WAV files worldwide, Id guess? :-D

Re: The text in Claude Code’s “Extended Thinking” output

#63

Post by dragonwriter »

anuramat wrote: post #16
ur-whale wrote: post #9

Earlier quoted context omitted.

> wdym? https://en.wikipedia.org/wiki/Economic_moat

how is summarized CoT a moat, and how is having the top 2 LLMs not a moat?

Not revealing actual thinking traces prevents mdoel distillation on yhe actual output (thinking traces are a key part of the output) which makes it harder for conpetitors to catch up (a moat).

Being currently in the lead in a category is not a moat,a moat is whatever creates a barrier to competitors catching up when you are in the lead. Merely being in the lead is not a moat except in a market with strong network externalities.

Re: The text in Claude Code’s “Extended Thinking” output

#64

Post by [deleted] »

irthomasthomas wrote: post #51

I won't use or recommend models with hidden reasoning, (thats all American models). It's too much of a risk and makes prompt optimization harder. Risky because it makes it possible for an attacker to prompt inject the reasoning chain to carry out a secret objective, and to hide that from the summary and output. Interleaved reasoning and function calling makes this even more dangerous. A model can call functions durin…

[deleted]

Re: The text in Claude Code’s “Extended Thinking” output

#65

Post by orangecat »

tsunamifury wrote: post #7

It’s not surprising than the Sota model makers core goal is to get user dependent while denying them increasing amounts of understanding of how it works to form a deeply unhealthy dependency. Tell me this. If you hired a junior engineer or designer who refused to explain their thinking on their code and how they solved for the spec what would you do? (That being said the reasoning output is still a summary of the Kvc…

* If you hired a junior engineer or designer who refused to explain their thinking on their code*

Any explanation that someone gives of their thinking process is necessarily lossy and likely partially confabulated.

Re: The text in Claude Code’s “Extended Thinking” output

#66

Post by dcrazy »

stingraycharles wrote: post #34
craigmart wrote: post #24

This is something we have known for a very long time, and companies are not trying to hide that either. They do it to avoid letting competitors train their models on the CoTs

Yes hasn’t this been around since Opus 4.6? I very much recall this change happening around January or February, and it was very explicitly to prevent distillation. Sonnet does not have this limitation. Fun fact: if you go back to the old school from 2 years ago and provide explicit CoT prompts, you get the full thinking prompts back again! So you disable thinking altogether, and instead make thinking part of the reg…

I thought this was considered best practice? I actually prefer it to exposed thought channel, much like how I would prefer a human answer with supporting logic instead of an explanation of their problem-solving approach.

Re: The text in Claude Code’s “Extended Thinking” output

#68

Post by bee_rider »

StizzurpXDD wrote: post #18

This is not just Anthropic. Almost all big AI companies, including OpenAI and Google, hide their model's actual reasoning. This is because revealing the raw reasoning exposes exactly how the AI processes information. These companies spend in huge amounts on R&D to develop a thinking process that is superior to their competition. Exposing those thinking mechanics to competitors would completely defeat the purpose of t…

Mistral displays some “thinking” text (in their basic online chat interface) in the thinking mode, do we know if those are the real tokens?

It’s quite interesting to read. I can’t imagine using a model like this without the ability to peek inside and see if it is getting stuck.

Re: The text in Claude Code’s “Extended Thinking” output

#69

Post by runeblaze »

tbh the summarized thinking with encrypted raw thinking is there for many purposes; it is there to:

1. make distillation much harder

2. safety: prevent modifications to the thinking leading to injection attacks.

3. also honestly sometimes the model raw thoughts can be deranged and is not a good user experience (consider the varied audience in the market, etc.)

also often the mass underestimate/the model makers over-estimate how people love distilling models

Re: The text in Claude Code’s “Extended Thinking” output

#70

Post by Roritharr »

irthomasthomas wrote: post #51

I won't use or recommend models with hidden reasoning, (thats all American models). It's too much of a risk and makes prompt optimization harder. Risky because it makes it possible for an attacker to prompt inject the reasoning chain to carry out a secret objective, and to hide that from the summary and output. Interleaved reasoning and function calling makes this even more dangerous. A model can call functions durin…

I've thought about the high-jacking of reasoning-chains as a potential vector, but never saw a proven implementation in american models since, from my understanding, all major vendors throw out the reasoning tokens between turns.
Post reply on HN