Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B is excellent, but it defaults to overthinking things

simonwillison.net

Post reply on HN
Print view View original

11–20 of 411 posts

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#11

Post by dofm »

cyanydeez wrote: post #8

--thinking-budget and --thinking-message is all you need in llamacpp to keep it progressing. the message can be some combination of tool calling, summarizing, etc. It's overthinking often is a bunch of recursion, so simply stopping t and redirecting is all you need to do. If someones building a harness for llamacpp, you can set this per message, so it's possible to dynamically control it by watching for the expansion…

Unfortunately in xhigh thinking it goes down rabbit holes in such an extreme depth-first way, that whenever you choose to cut it off, there is a very good chance it will not have got round to musing on even half of the prompt! It doesn’t really obviously loop in xhigh, so I am not sure if an “overthinking guard” proxy would have much to go on, but it does obsessively ruminate on edge cases. I have seen it overcomplicate simple code as a result even in my limited testing.

Probably the better solution if you want it to be quicker but still fairly thorough appears to be to configure reasoning effort instead of thinking budget. It seems to do very well still even on the Low setting; on the Medium setting it can get stuck in loops like 3.6 does.

I think xhigh reasoning effort was an absurd choice for a default, and so was not sorting out the chat template so LM Studio could offer the reasoning effort dropdown.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#12

Post by bitexploder »

cyanydeez wrote: post #8

--thinking-budget and --thinking-message is all you need in llamacpp to keep it progressing. the message can be some combination of tool calling, summarizing, etc. It's overthinking often is a bunch of recursion, so simply stopping t and redirecting is all you need to do. If someones building a harness for llamacpp, you can set this per message, so it's possible to dynamically control it by watching for the expansion…

Yeah, but be fair. Working with small models is a different ball game. Not all the batteries come included :)

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#15

Post by lostmsu »

andy99 wrote: post #2

The big problem with overthinking on a dense model is obviously the speed hit you take. Going from Qwen 35BA3B to 27B for me is about 7-8x slower (should be ~9x?). This makes me a lot less patient for useless thinking tokens. I’d want to compare this to the new Muse 30B model which is super terse and has a whole different way of thinking (no “Wait,”) and in my experiments was way more token efficient to the point tha…

Glimmer is stupider than 3.6 27B. You can't compare its speed to 3.8 and be done.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#16

Post by SwellJoe »

simonw wrote: post #6
SwellJoe wrote: post #4

This is true, but I think it understates the problem. I did a task I've done with a bunch of small models lately ( https://github.com/swelljoe/flar/pull/17 ), and it did an excellent job, the best of any self-hostable model. But, it took eleven (11!) hours on my dual GPU setup. It really chewed on it, and spent a lot of time checking and re-checking. It is by far the slowest model I've used for the task. GPT 5.5 did…

Was that with the default xhigh reasoning setting? I suggest trying again with reasoning set to low or turned off entirely.

Yes, default everything, no tuning, 8_K_XL Unsloth quantization on dual Radeon V620 GPUs (which aren't blazing, but faster than the Strix Halo).

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#18

Post by bellowsgulch »

cyanydeez wrote: post #8

--thinking-budget and --thinking-message is all you need in llamacpp to keep it progressing. the message can be some combination of tool calling, summarizing, etc. It's overthinking often is a bunch of recursion, so simply stopping t and redirecting is all you need to do. If someones building a harness for llamacpp, you can set this per message, so it's possible to dynamically control it by watching for the expansion…

This is definitely such a cool feature that I wish cloud providers would expose.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#19

Post by cyanydeez »

dofm wrote: post #11
cyanydeez wrote: post #8

--thinking-budget and --thinking-message is all you need in llamacpp to keep it progressing. the message can be some combination of tool calling, summarizing, etc. It's overthinking often is a bunch of recursion, so simply stopping t and redirecting is all you need to do. If someones building a harness for llamacpp, you can set this per message, so it's possible to dynamically control it by watching for the expansion…

Unfortunately in xhigh thinking it goes down rabbit holes in such an extreme depth-first way, that whenever you choose to cut it off, there is a very good chance it will not have got round to musing on even half of the prompt! It doesn’t really obviously loop in xhigh, so I am not sure if an “overthinking guard” proxy would have much to go on, but it does obsessively ruminate on edge cases. I have seen it overcomplic…

to the point though: most of that overthinking is useless if you have a proper redirect message. So setting arbitrary budget and getting it a good message will do the trick regardless of what type of thinking it's doing. The reason thinking seems to work is that it's just trying to find an optimum outside the local optimum, and the thinking trace helps find it.

The only think I could think that'd be better than the --reasoning-budget would bet a budget jitter just in case it really is repeating a pattern and you want to escape it arbitrarily, otherwise yes, it could keep looping if you're always cutting at the wrong time.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#20

Post by xscott »

It won't satisfy the people who just want to drop a model into their existing toolset and run, but I think there are a lot of ways to deal with this overthinking problem.

For instance, it's a step backward, but I put {"reasoning_effort":"none"} and led it by the nose:

   User: We're going to make .  Please create a plan, but do not write code yet.

   Agent: 

   User: Now please follow that plan and write the code.  No other chat.

   Agent:
Maybe this can be fixed with Jinja templates or something, or maybe it's a hack to your harness, but it shows you can get the model to reason reasonably.
Post reply on HN