Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The text in Claude Code’s “Extended Thinking” output

patrickmccanna.net

Post reply on HN
Print view View original

71–80 of 248 posts

Re: The text in Claude Code’s “Extended Thinking” output

#72

Post by visarga »

StizzurpXDD wrote: post #18

This is not just Anthropic. Almost all big AI companies, including OpenAI and Google, hide their model's actual reasoning. This is because revealing the raw reasoning exposes exactly how the AI processes information. These companies spend in huge amounts on R&D to develop a thinking process that is superior to their competition. Exposing those thinking mechanics to competitors would completely defeat the purpose of t…

When you export your personal data Google hides all model responses leaving just user messages. So it's even worse

Re: The text in Claude Code’s “Extended Thinking” output

#75

Post by JamesSwift »

Roritharr wrote: post #70
irthomasthomas wrote: post #51

I won't use or recommend models with hidden reasoning, (thats all American models). It's too much of a risk and makes prompt optimization harder. Risky because it makes it possible for an attacker to prompt inject the reasoning chain to carry out a secret objective, and to hide that from the summary and output. Interleaved reasoning and function calling makes this even more dangerous. A model can call functions durin…

I've thought about the high-jacking of reasoning-chains as a potential vector, but never saw a proven implementation in american models since, from my understanding, all major vendors throw out the reasoning tokens between turns.

> all major vendors throw out the reasoning tokens between turns

That would be surprising to me. The reasoning _is_ the model intelligence in a lot of respects, and so dropping those from the context would affect its output pretty significantly.

I assume that instead they just have a lot of guardrails in place and multiple runtime environments that an individual turns ping-pong between in order to dehydrate/rehydrate the reasoning to keep it hidden from the end user.

Re: The text in Claude Code’s “Extended Thinking” output

#78

Post by kapperchino »

irthomasthomas wrote: post #51

I won't use or recommend models with hidden reasoning, (thats all American models). It's too much of a risk and makes prompt optimization harder. Risky because it makes it possible for an attacker to prompt inject the reasoning chain to carry out a secret objective, and to hide that from the summary and output. Interleaved reasoning and function calling makes this even more dangerous. A model can call functions durin…

This agent I made can’t execute on the shell, can only edit the files within the project. Only works with rust atm though. https://github.com/Kapperchino/agent-joe

Re: The text in Claude Code’s “Extended Thinking” output

#79

Post by vesterde »

Roritharr wrote: post #70
irthomasthomas wrote: post #51

I won't use or recommend models with hidden reasoning, (thats all American models). It's too much of a risk and makes prompt optimization harder. Risky because it makes it possible for an attacker to prompt inject the reasoning chain to carry out a secret objective, and to hide that from the summary and output. Interleaved reasoning and function calling makes this even more dangerous. A model can call functions durin…

I've thought about the high-jacking of reasoning-chains as a potential vector, but never saw a proven implementation in american models since, from my understanding, all major vendors throw out the reasoning tokens between turns.

Gemini models return a thinking signature that you, I think, must send back when invoking further, so they seem to keep them?
Post reply on HN