Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B is excellent, but it defaults to overthinking things

simonwillison.net

Post reply on HN
Print view View original

51–60 of 411 posts

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#51

Post by hedgehog »

xscott wrote: post #20

It won't satisfy the people who just want to drop a model into their existing toolset and run, but I think there are a lot of ways to deal with this overthinking problem. For instance, it's a step backward, but I put {"reasoning_effort":"none"} and led it by the nose: User: We're going to make . Please create a plan, but do not write code yet. Agent: User: Now please follow that plan and write the code. No other chat…

To be fair a lot of models have quirks, I've never found a model swap that was transparent.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#52

Post by russfink »

jedbrooke wrote: post #50

I feel like the current “reasoning” that LLMs are doing has got to be a dead end eventually. Every time I have to read another answer with “but wait” and “Actually,” as they “reason” their way to a (sometimes) better answer, I feel like there’s got to be a way to just shortcut to the actual correct answer instead of burning all these token going in circles mimicking actual thought

It “thinks out loud” to populate its token space. Asking it to shortcut risks truncating that process.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#53

Post by jauntywundrkind »

russfink wrote: post #52
jedbrooke wrote: post #50

I feel like the current “reasoning” that LLMs are doing has got to be a dead end eventually. Every time I have to read another answer with “but wait” and “Actually,” as they “reason” their way to a (sometimes) better answer, I feel like there’s got to be a way to just shortcut to the actual correct answer instead of burning all these token going in circles mimicking actual thought

It “thinks out loud” to populate its token space. Asking it to shortcut risks truncating that process.

Yeah. It's "thinking" in absurd massive vectors. It needs to assess a couple to weigh out. That's the compression. That's the nature. It looks ridiculous when thinking traces render out such simple statements ('reassessing ..') but I expect this is far deeper an assessment than it can fully reflect to us on, and I expect its a huge part of their thinking.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#54

Post by NewJazz »

jedbrooke wrote: post #50

I feel like the current “reasoning” that LLMs are doing has got to be a dead end eventually. Every time I have to read another answer with “but wait” and “Actually,” as they “reason” their way to a (sometimes) better answer, I feel like there’s got to be a way to just shortcut to the actual correct answer instead of burning all these token going in circles mimicking actual thought

Hardcode their "thoughts" in your agents.md... But they might still reason through it anyway.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#55

Post by nojs »

jatora wrote: post #38

All current era models overthink as it's a product of their RL incentives (or distillation of models with them...) From my reading of the Fable 5 and Opus 5 System cards, my reconstruction is something like: Finish the task → make externally observable evidence that it is finished → check your own work → fix problems → don't stop prematurely → satisfy the evaluator comprehensively. That is fantastic for SWE benchmark…

For a model this small it’s also a way of trading supply-constrained VRAM for inference time, which for self hosting consumers (and probably Chinese companies subject to export controls) is mostly good.

I can imagine a curve where for a given level of “intelligence” you either need model size, or inference time (“test time compute”), and can somewhat trade one for the other.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#56

Post by matheusmoreira »

Am I the only one who enjoys it when LLMs overthink everything?

Opus 4.8 would spend like 10 minutes thinking and then go out there and do an excellent job. Only Fable 5 seems to be smart enough to just know everything it needs to immediately start working without any reasoning or verification. Opus 5 tries to be relentless like Fable, but it's not as smart as Fable and I have to constantly challenge and correct its unfounded assumptions. Sol is somewhere between Fable and Opus 5, it's smart but it's not Fable, it keeps making assumptions that I have to correct.

After trying all these models, I find that I miss Opus 4.8's overthinking. Sure it's slow, but it actually gets things right.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#57

Post by NitpickLawyer »

simonw wrote: post #5
andy99 wrote: post #2

The big problem with overthinking on a dense model is obviously the speed hit you take. Going from Qwen 35BA3B to 27B for me is about 7-8x slower (should be ~9x?). This makes me a lot less patient for useless thinking tokens. I’d want to compare this to the new Muse 30B model which is super terse and has a whole different way of thinking (no “Wait,”) and in my experiments was way more token efficient to the point tha…

Comparing with Muse Glimmer is a good idea. I ran the same exact HTML tool generating prompt against both Glimmer 30B and Qwen 3.8 27B. Results: Qwen: https://gist.github.com/simonw/121ad098860028b2fab603fa12da1... - 17,576 reasoning tokens, produced this HTML result: https://static.simonwillison.net/static/2026/qwen-over-think... Glimmer: https://gist.github.com/simonw/51e8ddb2ee597a5005fa63bd4927d... 1,021 reasonin…

Yesterday I tried both as well. I do a quick "explain this repo" + "any security issues" convo to do a "vibe check" on the models and make sure everything works w/ serving and harness. Both qwen and glimmer explained it pretty well, and both accepted the security question without any issues, flagged a few things left there on purpose (hardcoded tokens, single auth, no logs, etc).

I like the style of glimmer more. Much terser language, no adjectives, no fluffy claude-like language. ("Images are written to...", "Tasks are stored in SQLite...", "Docker image is built from ...")

In contrast, qwen is a bit more flowery. ("Unbounded image processing / resource exhaustion — preprocess() opens whatever was downloaded with no size/dimension/format validation before the VAE encodes it..." , "SQLite as a queue — fine at this scale, but...", "Debug info leakage — exceptions are re-raised as...".

But both flagged pretty much the same stuff, just ordered / styled differently. Mighty impressive understanding for a thing that I can run locally. Qwen served in fp8 w/ full kv cache, glimmer in w4a16 (the fp8 weights wouldn't serve for whatever reason), both at full supported context in 48GB of VRAM.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#58

Post by matheusmoreira »

nharziro wrote: post #25

I do agree that Qwen 3.8 27B is excellent but slow and very token inefficient. My benchmark places it near opus 4.6 and codex 5.3 performance. 3.6 27B couldn't even complete the benchmark. Please see below for details: https://gist.github.com/nharziro/aed0c364ce2f295a493494c6f1b...

Opus 4.6 performance with a local model that can be hosted on consumer hardware is an incredible result!!

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#59

Post by bblb »

bogzz wrote: post #9
andy99 wrote: post #2

The big problem with overthinking on a dense model is obviously the speed hit you take. Going from Qwen 35BA3B to 27B for me is about 7-8x slower (should be ~9x?). This makes me a lot less patient for useless thinking tokens. I’d want to compare this to the new Muse 30B model which is super terse and has a whole different way of thinking (no “Wait,”) and in my experiments was way more token efficient to the point tha…

I love reading Glimmer's "thoughts". Why use many word when few do trick?

Me machine, no human. Why waste token.

Do fast, deliver.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#60

Post by teravor »

when you distill a thinking LLM past its capacity it will default to overthinking because during training that was the only way for a chance at a reward on many tasks.

you can generally avoid this if you specialize it on a domain that is within its capacity.

Post reply on HN