Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B is excellent, but it defaults to overthinking things

simonwillison.net

Post reply on HN
Print view View original

341–350 of 411 posts

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#341

Post by dofm »

bitexploder wrote: post #49
dofm wrote: post #26

Earlier quoted context omitted.

Unfortunately in xhigh reasoning effort it will burn through 2K tokens before it has even finished its bullet point overview. It really is intense and obsessive. You might need ten times more! Your strategy would likely help in medium reasoning effort (because there it gets caught up in the very typical Qwen looping). Not seen looping in the “low” reasoning effort mode.

I have been using Muse Glimmer for a few days instead of A3B. It gets the job done quicker than A3B despite being several times slower.

Yes — I just found out that you can set reasoning level in the prompt — like with Qwen 3.8 27B it is actually really pretty solid at "Reasoning level: low".

Ten to thirteen tokens per second on my M1 Max (might be some room to improve this) but it indeed solved as fast as the Qwen 35B. 40 seconds faster on one of my tests that involves three steps.

This is very striking.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#342

Post by lta »

nsingh2 wrote: post #64
jedbrooke wrote: post #50

I feel like the current “reasoning” that LLMs are doing has got to be a dead end eventually. Every time I have to read another answer with “but wait” and “Actually,” as they “reason” their way to a (sometimes) better answer, I feel like there’s got to be a way to just shortcut to the actual correct answer instead of burning all these token going in circles mimicking actual thought

One line of evolution seems to be toward some form of latent-space reasoning, as in [1]. Natural language seems like a relatively low-bandwidth channel for intermediate reasoning. [1] https://github.com/sapientinc/HRM-Text

While I was reading the top level message I was thinking on something wrong those lines. That being said I think it's somewhat useful to be able to see the reasoning in a meaningful format. There's already enough hidden layers of unfathomable meaning.

What if instead of a latent they used at a shorter form of note-taking-like reasoning, using more symbol to achieve a denser CoT. We'd get the best of both worlds. WDYT ?

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#343

Post by pessimizer »

qeternity wrote: post #272
janalsncm wrote: post #159

Earlier quoted context omitted.

I think the point is that if people are able to run inference on their laptops batch size efficiency won’t matter. And before that, businesses will be able to get decent results with dedicated inference hardware.

Jevons paradox: large purpose-fit data centers increase efficiency such that you can use AI in more places, and use more tokens for those tasks. The future is not a single chat bot session of bs=1. The future is many agents performing many tasks in parallel for a single user. Large GPU clusters will always have the edge in efficiency.

> The future is many agents performing many tasks in parallel for a single user. Large GPU clusters will always have the edge in efficiency.

Agentic AI has pluses and minuses for cloud efficiency. The plus is that usage could be very bursty, but the minus is that agents will more fully utilize a local system. The main disadvantage of local AI is that you would be paying a large amount for a system mostly doing nothing. If it's constantly working on different projects and integrating that data, you get use out of every penny that you spent. Every GPU you added would instantly make the thing smarter.

What's more, your local AI could offload an agent to the cloud if it needed to. It could do this rationally, based on your personal desire for privacy.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#344

Post by CamperBob2 »

Bender wrote: post #339
chvid wrote: post #66

“The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.” I think that should be the blinking headline - this shows what can be done with consumer hardware.

Do these self hosted models avoid "protecting the user" or protecting big businesses? In other words can I just ask it any question and if it has the answer, I will get an answer rather than telling me it can't answer the question? I ask because Claude is fun for rewriting abandoned code and I am not a proper developer so it's been great for me. Claude refuses to answer questions about science and medicine that stray…

Out of the box, open-weight models have guardrails similar to the rest. But unlike the closed models they can be 'abliterated' with varying degrees of success. If you run an aggressive Heretic abliteration of Qwen 3.6 27B you will not generally experience either refusals or an obvious degradation in quality.

I personally like the https://huggingface.co/HauhauCS version of Qwen 27B from a purely subjective point of view as a user, but that particular one has come under criticism for reasons that don't necessarily affect its quality or usability.

Some of the larger models have also undergone similar treatment, but it's less common.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#345

Post by ipdashc »

mattmaroon wrote: post #318
ipdashc wrote: post #202

Earlier quoted context omitted.

Makes sense. Thanks for the post!

Do you work in the space?

Nope, not at all. Was just curious what you were using it for that was so unrelated to computers, but what you said makes a lot of sense

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#346

Post by simonw »

Anonyneko wrote: post #338
blagui wrote: post #22

You have 4 thinking levels. You can disable it. It's well known issue in Qwen, previous releases I would disable it by default. Also xhigh seem a new thing.

So it can be disabled on 27B after all? I haven't downloaded 3.8 27B yet, but the announcement made it seem like only the hosted Max can disable thinking, and with the open weights you could only select between provided reasoning efforts.

I've been passing "reasoning": { "effort": "none" } to LM Studio's Responses API and the resulting response doesn't include any reasoning chunks, so that seems to work.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#347

Post by simonw »

naasking wrote: post #336

To what extent can the overthinking be reduced by a better system prompt? For instance, telling it not to overthink, embellish or snazz anything up, and just directly deliver the most what was requested without in the plainest form to build on in future iterations.

I tried "render an svg of five intersecting squares. don't overthink this" and... it overthought it: https://static.simonwillison.net/static/2026/cors-chat.webp

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#348

Post by cjbprime »

> strong recommendation: ignore that default. Run Qwen 3.8 27B on low or even no reasoning levels at first. It’s a great model, but wow that default setting is a bad place to start.

Has anyone tried asking the model to choose and emit the most appropriate reasoning level for each prompt, as the first part of answering it?

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#349

Post by naasking »

simonw wrote: post #347
naasking wrote: post #336

To what extent can the overthinking be reduced by a better system prompt? For instance, telling it not to overthink, embellish or snazz anything up, and just directly deliver the most what was requested without in the plainest form to build on in future iterations.

I tried "render an svg of five intersecting squares. don't overthink this" and... it overthought it: https://static.simonwillison.net/static/2026/cors-chat.webp

I don't think "don't overthink" is a useful guide, it's too ambiguous which leads to the model overthinking how to interpret the advice to not overthink.

I think guidelines have to be specific, such as "provide only the specific properties specified, and if other properties are required but are not specified, then ask for clarification or pick the simplest default possible". Even that's ambiguous on "simplest default possible".

I would be more specific in specific domains. The circle example from the article is clear that it defaults to elaborate stylistic choices rather than simple colours and shapes, so the prompt would have to specify "use only simple colours and the simplest possible shape that fulfills the request".

Post reply on HN