Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B is excellent, but it defaults to overthinking things

simonwillison.net

Post reply on HN
Print view View original

201–210 of 411 posts

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#201

Post by robkop »

cchance wrote: post #74
dannyw wrote: post #70

Earlier quoted context omitted.

The recent 'Stolen Thoughts'[1] paper shows many excerpts of private reasoning for frontier models. For a complex maths problem, Sol reasoned in 367 tokens before working: We need solve. Need interpret no person sits next to two other people = among binary string length16 weight8, no occupied chair whose both neighbors occupied, equivalently ab 111 substring. Endpoints cannot have two neighbors anyway; only avoid 111…

All these weird partial language thought patterns im surprised none of the teams have taught the models to think in something like court stenography or some very dense pattern (i mean they even tried caveman language at one point)

One of the main goals at the moment is to keep thinking human legible. You can imagine how much harder it’d be to do root cause analysis on the recent OpenAI event if we couldn’t even tell what they’re thinking.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#202

Post by ipdashc »

mattmaroon wrote: post #133
redrove wrote: post #129

Earlier quoted context omitted.

I would also be interested in what you use AI for. Is it marketing? bureaucracy?

Yes. Bureaucracy for sure when I throw festivals. ChatGPT pulls permits for me. (I think that uses a lot of tokens because of the browser control?) Manages the admin side along with some tools I built in Lovable via mcp servers. Marketing definitely. My food truck side is relatively high volume and it manages my kitchen and warehouse side, basically generating all of the instructions my employees follow, managing and…

Makes sense. Thanks for the post!

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#204

Post by Phemist »

CamperBob2 wrote: post #113
jedbrooke wrote: post #50

I feel like the current “reasoning” that LLMs are doing has got to be a dead end eventually. Every time I have to read another answer with “but wait” and “Actually,” as they “reason” their way to a (sometimes) better answer, I feel like there’s got to be a way to just shortcut to the actual correct answer instead of burning all these token going in circles mimicking actual thought

Chain-of-thought output shouldn't be taken literally. The tokens are a substrate for computation, not necessarily evidence that the model is wasting time and electricity by gratuitously second-guessing itself over and over. You can see evidence of this phenomenon in models dating back to the OG Deepseek R1. It was common to see the model talk itself out of the correct solution in the block, or fail to reach it at all…

I am interested however in why fine-tuning on reasoning traces of a frontier model is such an effective way of improving an (open-weight) base model. See e.g. https://huggingface.co/hesamation/Qwen3.6-35B-A3B-Claude-4.6...

I can see the reasoning being a substrate for computation, but in which space should we interpret this computation to be happening? The vector representations of individual tokens are completely different (and even the way the reasoning traces are broken up into tokens will be pretty different) between Qwen and Claude e.g.. The only way I can see this being effective (which it is) is thus that we SHOULD interpret the model to be "computing" in natural language and thus we can indeed take the chain-of-though somewhat literally.

The Deepseek R1 behaviour you describe is from a model from january last year, are you sure this is not pathological behaviour rather than an indication of the reasoning not needing to be taken literally?

I do however agree with the point that it is not necessarily a dead-end. That Qwen loops almost at an OCD-like level, but retains accuracy on the times it does answer, shows that. Yes ideally it loops less, but I am for now happy to accept that this is what it takes to run models locally. At least it is available for our inspection.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#205

Post by niek_pas »

nsingh2 wrote: post #64
jedbrooke wrote: post #50

I feel like the current “reasoning” that LLMs are doing has got to be a dead end eventually. Every time I have to read another answer with “but wait” and “Actually,” as they “reason” their way to a (sometimes) better answer, I feel like there’s got to be a way to just shortcut to the actual correct answer instead of burning all these token going in circles mimicking actual thought

One line of evolution seems to be toward some form of latent-space reasoning, as in [1]. Natural language seems like a relatively low-bandwidth channel for intermediate reasoning. [1] https://github.com/sapientinc/HRM-Text

What does ‘latent’ mean in this context?

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#206

Post by tapland »

shafyy wrote: post #200
coldtea wrote: post #179

Earlier quoted context omitted.

Not if an LLM over chat can fool most people they're talking to a human (which it can), where the TI-99 speech synthesizer voice absolutely can not.

Can it? I feel like I instantly recognize if I am chatting with an LLM or a human

Can't even tell if you're real or a bot by reading one comment.

Great times!

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#208

Post by IMTDb »

shafyy wrote: post #200
coldtea wrote: post #179

Earlier quoted context omitted.

Not if an LLM over chat can fool most people they're talking to a human (which it can), where the TI-99 speech synthesizer voice absolutely can not.

Can it? I feel like I instantly recognize if I am chatting with an LLM or a human

Emphasis on "feel"

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#209

Post by piva00 »

shafyy wrote: post #200
coldtea wrote: post #179

Earlier quoted context omitted.

Not if an LLM over chat can fool most people they're talking to a human (which it can), where the TI-99 speech synthesizer voice absolutely can not.

Can it? I feel like I instantly recognize if I am chatting with an LLM or a human

I was quite surprised on how difficult it is to tell when chatting with an uncensored LLM a friend is running (it's too big to run on any of my computers but he got some B200s). You can input your own "system prompt" to make it behave like a normal internet user and the prose writes very similarly to internet comments with none of the LLMisms from ChatGPT, Claude, Grok, etc.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#210

Post by zahlman »

coldtea wrote: post #179
bitwize wrote: post #141

Earlier quoted context omitted.

Indeed. LLMs resemble human intelligence in more or less the same way that the output of the TI-99/4A speech synthesizer resembles a human voice.

Not if an LLM over chat can fool most people they're talking to a human (which it can), where the TI-99 speech synthesizer voice absolutely can not.

> Not if an LLM over chat can fool most people they're talking to a human (which it can)

I keep hearing this claim, and yet I keep seeing LLM output which is trivially distinguished from human writing. I really can't understand how this gap persists; but then, there seem to have been at least some people who couldn't sniff out ELIZA, back in the day, too.

Post reply on HN