Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B is excellent, but it defaults to overthinking things

simonwillison.net

Post reply on HN
Print view View original

21–30 of 411 posts

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#21

Post by dofm »

cyanydeez wrote: post #19
dofm wrote: post #11

Earlier quoted context omitted.

Unfortunately in xhigh thinking it goes down rabbit holes in such an extreme depth-first way, that whenever you choose to cut it off, there is a very good chance it will not have got round to musing on even half of the prompt! It doesn’t really obviously loop in xhigh, so I am not sure if an “overthinking guard” proxy would have much to go on, but it does obsessively ruminate on edge cases. I have seen it overcomplic…

to the point though: most of that overthinking is useless if you have a proper redirect message. So setting arbitrary budget and getting it a good message will do the trick regardless of what type of thinking it's doing. The reason thinking seems to work is that it's just trying to find an optimum outside the local optimum, and the thinking trace helps find it. The only think I could think that'd be better than the -…

> The reason thinking seems to work is that it's just trying to find an optimum outside the local optimum, and the thinking trace helps find it.

Yes, I think I finally have an intuitive sense for that. But surely on a longer prompt it is still better for the final response if the thinking has at least brushed past all of the prompt?

One of the things I witnessed with xhigh is that while the thinking trace starts out intending an overview of the prompt, it actually can go fully down a rabbit hole off one of the first two or three bullet points even when it was seemingly intending not to.

It’s basically a lot like me. Gets sidetracked by the interesting bits.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#23

Post by fermuch »

SwellJoe wrote: post #4

This is true, but I think it understates the problem. I did a task I've done with a bunch of small models lately ( https://github.com/swelljoe/flar/pull/17 ), and it did an excellent job, the best of any self-hostable model. But, it took eleven (11!) hours on my dual GPU setup. It really chewed on it, and spent a lot of time checking and re-checking. It is by far the slowest model I've used for the task. GPT 5.5 did…

xhigh tells it to overthink and re check everything. Low tells it to only do the minimum thinking necessary. I would suggest to give qwen medium which doesn't inject any thinking directives into it and also to give as much context as you can, ideally around 500k tokens or even 1M if you can. Big complex tasks like these make the model hit the compaction trigger a lot and they end up re thinking the same thing several times in my experience.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#24

Post by doginasuit »

To be fair, Opus 5 overthinks things on a regular basis. I interact with the LLM almost entirely through the prompt interface vs. some agentic harness, so I have a lot of granular exposure to its reasoning. For almost every code analysis, it flags all the important issues and at least one non-issue. It suggests some impractical and unnecessary fix for the non-issue that would categorically be a regression.

I've learned that medium effort can improve the outcome relative to higher settings. But I suspect the phenomenon is an artifact of a misguided effort to fix inherent LLM limitations. At least some of its reasoning will miss the target, and more bad reasoning is not the remedy.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#25

Post by nharziro »

I do agree that Qwen 3.8 27B is excellent but slow and very token inefficient. My benchmark places it near opus 4.6 and codex 5.3 performance. 3.6 27B couldn't even complete the benchmark. Please see below for details:

https://gist.github.com/nharziro/aed0c364ce2f295a493494c6f1b...

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#26

Post by dofm »

bitexploder wrote: post #10
javchz wrote: post #7

I wonder if this can be fixed with LORAs.

I had to fix this on 35B A3B -- I have a proxy that just shuts it down if it gets to 2K thinking tokens and injects something like "We have thought enough, let's begin working." and it almost always finishes the turn then. It rarely needs more than 2K thinking tokens and if it does there is always next turn. I would need to see what 27B is actually doing, but these smaller Qwen models seem prone to this.

Unfortunately in xhigh reasoning effort it will burn through 2K tokens before it has even finished its bullet point overview. It really is intense and obsessive. You might need ten times more!

Your strategy would likely help in medium reasoning effort (because there it gets caught up in the very typical Qwen looping).

Not seen looping in the “low” reasoning effort mode.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#27

Post by dofm »

blagui wrote: post #22

You have 4 thinking levels. You can disable it. It's well known issue in Qwen, previous releases I would disable it by default. Also xhigh seem a new thing.

Yes. Though the chat template doesn’t tell LM Studio to offer the little dropdown. You can bodge the template in the load parameters.

Unsloth Studio / Desktop has it working really well with their version of the weights.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#28

Post by dofm »

bogzz wrote: post #9
andy99 wrote: post #2

The big problem with overthinking on a dense model is obviously the speed hit you take. Going from Qwen 35BA3B to 27B for me is about 7-8x slower (should be ~9x?). This makes me a lot less patient for useless thinking tokens. I’d want to compare this to the new Muse 30B model which is super terse and has a whole different way of thinking (no “Wait,”) and in my experiments was way more token efficient to the point tha…

I love reading Glimmer's "thoughts". Why use many word when few do trick?

It’s also a little bit snarky, almost. The stuff it thought during the car wash puzzle made me laugh.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#29

Post by dofm »

LoganDark wrote: post #14

I hope Apple does end up moving to HBM. Unified memory has been a huge godsend, but the low memory bandwidth is just such a killer. Even/especially on M5, where the available compute is starting to starve incredibly badly on ML workloads.

AFAIK that is initially only for the iPhone?
Post reply on HN