i wonder if it's about protecting it from extraction/distillation or if it's about not having to answer for surface that hasn't been properly vetted for public consumption. (ie, is someone going to sue them or complain or write blog posts because the thinking has transient things that people don't like where the final result is what is actually vetted?)
The text in Claude Code’s “Extended Thinking” output
161–170 of 248 posts
Re: The text in Claude Code’s “Extended Thinking” output
#162Earlier quoted context omitted.
It's similar to the process that transformers use when you ask them to do arithmetic without tools, I think. Some CoT tokens must be emitted up front for use as a computational substrate, but exactly what tokens they are isn't necessarily important or relevant to the final answer. And when that answer is returned, it may not be possible to tell what the actual reasoning process looked like behind the scenes. It only…
> Proper distillation requires access to the logits Why do you need logits? Can't you just train on cross-entropy loss of the model against the hard decision, like you do in regular pretraining? There are definitely current-gen open-weight models (Step 3.7 Flash is one) that refer to themselves as an OpenAI model in CoT, but not in the final response.
How do I get that loss, though, without the softmax inputs?
Re: The text in Claude Code’s “Extended Thinking” output
#163I won't use or recommend models with hidden reasoning, (thats all American models). It's too much of a risk and makes prompt optimization harder. Risky because it makes it possible for an attacker to prompt inject the reasoning chain to carry out a secret objective, and to hide that from the summary and output. Interleaved reasoning and function calling makes this even more dangerous. A model can call functions durin…
I don't think there can be tool calls inside the obfuscated reasoning blocks. I mean, in order for those function calls to be evaluated client-side, that thinking stream would have to be decrypted on the client side at some point, which would defeat the purpose of obfuscating it the way they do. If you mean the function calls might happen server side, there is nothing preventing the server from doing it and hiding it…
There is server-side tool calling, such as gemini using google search and gdrive.
Also, many clients minimize the code block by default so you mostly scan the summaries. Poisoned client side code could easily escape your attention.
Re: The text in Claude Code’s “Extended Thinking” output
#164This is not just Anthropic. Almost all big AI companies, including OpenAI and Google, hide their model's actual reasoning. This is because revealing the raw reasoning exposes exactly how the AI processes information. These companies spend in huge amounts on R&D to develop a thinking process that is superior to their competition. Exposing those thinking mechanics to competitors would completely defeat the purpose of t…
The cynic in me is wondering whether it's more about how revealing how the sausage is made might bring bad publicity.
[dead]
Re: The text in Claude Code’s “Extended Thinking” output
#165Earlier quoted context omitted.
Yes hasn’t this been around since Opus 4.6? I very much recall this change happening around January or February, and it was very explicitly to prevent distillation. Sonnet does not have this limitation. Fun fact: if you go back to the old school from 2 years ago and provide explicit CoT prompts, you get the full thinking prompts back again! So you disable thinking altogether, and instead make thinking part of the reg…
I thought this was considered best practice? I actually prefer it to exposed thought channel, much like how I would prefer a human answer with supporting logic instead of an explanation of their problem-solving approach.
[flagged]
Re: The text in Claude Code’s “Extended Thinking” output
#166Earlier quoted context omitted.
More to the point - if they expose their model's "thinking" inference, competitors can train on that to replicate the results. If they postprocess that content, e.g. by summarizing it, it's no longer as useful to competitors.
Exactly. Google won't like it if they spend millions to make Gemini 3.5 Pro's thinking the best in the world, only for Anthropic or OpenAI to copy it by just seeing the thinking process.
Copying for me, not for thee
Re: The text in Claude Code’s “Extended Thinking” output
#167Earlier quoted context omitted.
Imagine if their target customers, C-suite execs looking to replace workers, knew how unlike "thinking" this process actually was! we can't have that.
To be honest I'm not sure if many C-suite execs have a good idea of what "thinking" looks like inside in the first place, in the sense of focused mental activity aimed at solving of a hard logical or technical problem.
How did they became C-suite execs in the first place, if they don't know how to work on problems?
Re: The text in Claude Code’s “Extended Thinking” output
#168Earlier quoted context omitted.
"Your text batch moved the weights away from the final values. Your contribution is negative."
Where do I collect the $0.00000012 antidollars owed to me by OpenAI for my valuable inputs? Slightly more seriously, you could perhaps make an argument that, just like weight decay, an apparent "anti-contribution" moves the learning trajectory along, and helps the network settle into a more optimal basin eventually. That way, my contribution is still valuable on the net, and I'm owed $0.00000003 positive dollars inst…
>you could perhaps make an argument that, just like weight decay, an apparent "anti-contribution" moves the learning trajectory along
Was that not the joke?
Re: The text in Claude Code’s “Extended Thinking” output
#169Earlier quoted context omitted.
OAI is now implementing encrypted CoT that you can store and pass back between turns (harness call), so new models have it https://developers.openai.com/api/docs/guides/reasoning#encr...
You could also use the responses api which stores all message contents (including reasoning) on OAI servers. This has been possible for quite a while now. Encryption is only necessary if you really care about local storage (which is different from privacy concerns, because the data gets sent to their servers anyway).
well the encryption part is also mostly about OAI wanting to avoid others to distill from their COT/reasoning traces, since this is not ever displayed to devs or final users, and as you say lives on their servers.
but yes you're correct on the responses api already baking it in too
supposedly keeping these between tool calls should help the model reason and have better overall outputs etc
Re: The text in Claude Code’s “Extended Thinking” output
#170I feel like I get a lot of what this article presents as "hidden" by using this process: - "Read `description` and create a specification, implementation guide, and checklist." - "Ask clarifying questions. If any of those questions has a clear best recommendation, please select that yourself and record that in "autorecommendations.md". - "Have codex and antigravity review each of these and work to consensus." These a…
What are you using exactly to have claude code natively interact with codex and antigravity? Mind sharing your prompts?
Not at all, I do have a meeting here, I'll try to get it up in around 2h.