Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B is excellent, but it defaults to overthinking things

simonwillison.net

Post reply on HN
Print view View original

31–40 of 411 posts

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#31

Post by logicallee »

bitexploder wrote: post #10
javchz wrote: post #7

I wonder if this can be fixed with LORAs.

I had to fix this on 35B A3B -- I have a proxy that just shuts it down if it gets to 2K thinking tokens and injects something like "We have thought enough, let's begin working." and it almost always finishes the turn then. It rarely needs more than 2K thinking tokens and if it does there is always next turn. I would need to see what 27B is actually doing, but these smaller Qwen models seem prone to this.

>I had to fix this on 35B A3B -- I have a proxy that just shuts it down if it gets to 2K thinking tokens and injects something like "We have thought enough, let's begin working." and it almost always finishes the turn then.

that is amazing, thanks for sharing.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#32

Post by syntaxing »

SwellJoe wrote: post #16
simonw wrote: post #6

Earlier quoted context omitted.

Was that with the default xhigh reasoning setting? I suggest trying again with reasoning set to low or turned off entirely.

Yes, default everything, no tuning, 8_K_XL Unsloth quantization on dual Radeon V620 GPUs (which aren't blazing, but faster than the Strix Halo).

What speed do you get on this setup? Im tempted to use the same GPU.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#33

Post by refactor_master »

RachelF wrote: post #30

To me, the amazing thing is that we now have local models that rival the reasoning of high end models from about a year ago. I hope this trend continues.

Unlike cloud infra in general which offers things like automatic backups, regional redundancy, and effectively unlimited scalability, it seems like the value proposition of cloud LLM gets ever shakier.

* Many businesses don't need frontier level intelligence anyway.

* It's completely stateless. If your local LLM machine catches fire? Nothing was lost. Buy another.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#34

Post by xlayn »

I have this branch of llama.cpp that among other things (like patching the template to not break the kv cache, and saving conversations to disk so you can resume quickly days after) also accept the reasoning effort flag here https://github.com/alainnothere/llama.cpp/tree/disk-cache-ev...

I did testing and the reasoning effort can be set per message, I was not aware of the option of none mentioned by @xscott, I tested but didn't see any change, I think there are just 3 values, xhigh, medium and low as per https://huggingface.co/Qwen/Qwen3.8-27B-FP8 , I did testing and the thing can do it's "I'll speak 10 million words to myself to ensure I'm not missing something" and then switch to a faster model, then switch... I did a test and the thing keep coherence and follow it's train of though-kens, you can see the result here... https://github.com/alainnothere/llama.cpp/blob/disk-cache-ev...

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#35

Post by adam_arthur »

xscott wrote: post #20

It won't satisfy the people who just want to drop a model into their existing toolset and run, but I think there are a lot of ways to deal with this overthinking problem. For instance, it's a step backward, but I put {"reasoning_effort":"none"} and led it by the nose: User: We're going to make . Please create a plan, but do not write code yet. Agent: User: Now please follow that plan and write the code. No other chat…

Yes, if you set reasoning to none you can force the granularity of the thinking.

It will actually adhere to your request for e.g. 3 sentences max.

Thinking mode will override any instructions in the prompt (at least for other models in my experience).

Of course this will probably hurt performance, but works great for easy tasks that you know are trivial. Tons of pipeline, image recognition etc use cases where this works well.

I'd be curious to see Qwen 3.8 27B low thinking benchmarks though.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#36

Post by dofm »

simonw wrote: post #6
SwellJoe wrote: post #4

This is true, but I think it understates the problem. I did a task I've done with a bunch of small models lately ( https://github.com/swelljoe/flar/pull/17 ), and it did an excellent job, the best of any self-hostable model. But, it took eleven (11!) hours on my dual GPU setup. It really chewed on it, and spent a lot of time checking and re-checking. It is by far the slowest model I've used for the task. GPT 5.5 did…

Was that with the default xhigh reasoning setting? I suggest trying again with reasoning set to low or turned off entirely.

You now have me testing it with reasoning turned off, which I have never bothered much with on any other local models because it's rarely worth it.

The result appears to be almost as good as Qwen 3.6 35B A3B on medium thinking mode.

It second-guesses a little, it gives broader/more speculative answers, of course, and it missed the nuance of one of my prompts, but this gives me a lot more confidence that the Low reasoning effort is going to be as good as they say, and perhaps in some cases non-thinking looks like it would be enough.

Really useful, thanks.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#38

Post by jatora »

All current era models overthink as it's a product of their RL incentives (or distillation of models with them...)

From my reading of the Fable 5 and Opus 5 System cards, my reconstruction is something like:

Finish the task → make externally observable evidence that it is finished → check your own work → fix problems → don't stop prematurely → satisfy the evaluator comprehensively.

That is fantastic for SWE benchmarks and autonomous agents. It also naturally creates pathologies:

under-answering is expensive; over-answering is cheap.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#39

Post by Gracana »

xlayn wrote: post #34

I have this branch of llama.cpp that among other things (like patching the template to not break the kv cache, and saving conversations to disk so you can resume quickly days after) also accept the reasoning effort flag here https://github.com/alainnothere/llama.cpp/tree/disk-cache-ev... I did testing and the reasoning effort can be set per message, I was not aware of the option of none mentioned by @xscott, I tested…

What’s that about the template breaking the kv cache?
Post reply on HN