Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B is excellent, but it defaults to overthinking things

simonwillison.net

Post reply on HN
Print view View original

161–170 of 411 posts

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#161

Post by qlte »

dannyw wrote: post #112
solarkraft wrote: post #109

I find that a lot of the recent allegedly great open models are cranking their reasoning way further than I find reasonable for interactive use. I’m writing this while waiting for the new Deepseek V4 Flash to finish its task, which is taking way longer than the older version. What gets reported is always the benchmark result, but rarely the real-world trade-off made to achieve it. That’s an obvious incentive for the…

The model itself is excellent, the defaults are bad. As Simon and other pointed out, medium is great. Reminds me of Gemma4 and the official (or at least popularly used around launch) Jinja templates being wrong and broken for tool calling.

Seems less like "bad"/broken defaults and more defaults tuned to the max for benchmarks.

All the positive PR from "Opus 4.6 level" online buzz is well worth the minor annoyance from taking a half hour to solve a simple problem since a user just needs to turn down the reasoning knob if it bothers them.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#162

Post by SimplyUnknown »

XCSme wrote: post #154
monster_truck wrote: post #150

Earlier quoted context omitted.

$0.286/kWh is a ridiculous amount of money to pay for power. That's more than double the regional residental average here! If I ever found myself in this situation I would much rather just rent cards from hotasile and run open models instead of giving OAI money and playing reset bingo

I removed the extra links to sources for the electricity prices, but that's the average cost in EU, where I live. https://ec.europa.eu/eurostat/web/products-eurostat-news/w/d...

Indeed, EPEX spot prices are between €0.30 and €0.39 today including taxes where I live. I've seen them go up to €1.10 during peak demand.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#163

Post by theshrike79 »

AgentMatt wrote: post #158
monster_truck wrote: post #150

Earlier quoted context omitted.

$0.286/kWh is a ridiculous amount of money to pay for power. That's more than double the regional residental average here! If I ever found myself in this situation I would much rather just rent cards from hotasile and run open models instead of giving OAI money and playing reset bingo

Ridiculous? Wow. I'm paying ~$0.4/kWh in western Europe...

The current spot price in northern Europe is 0.025€/kWh, yesterday it was 0.007€/kWh :D

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#164

Post by vanviegen »

xscott wrote: post #20

It won't satisfy the people who just want to drop a model into their existing toolset and run, but I think there are a lot of ways to deal with this overthinking problem. For instance, it's a step backward, but I put {"reasoning_effort":"none"} and led it by the nose: User: We're going to make . Please create a plan, but do not write code yet. Agent: User: Now please follow that plan and write the code. No other chat…

My impression is that when you allow the model to use internal thinking as opposed to asking it to output its thinking first, it's more likely to backtrack when I detects a flaw in its plan. Said otherwise: once producing user-facing output it seems to lock into an approach, for better or worse.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#165

Post by spider-mario »

bakraman wrote: post #146
simonw wrote: post #83

Earlier quoted context omitted.

It should just about be usable in 32GB.

RAM is never the issue, it's always the compute power

RAM is not “never” the issue. My iPhone and MacBook Air could both run larger and more capable models if they had more RAM.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#166

Post by manmal »

refactor_master wrote: post #33
RachelF wrote: post #30

To me, the amazing thing is that we now have local models that rival the reasoning of high end models from about a year ago. I hope this trend continues.

Unlike cloud infra in general which offers things like automatic backups, regional redundancy, and effectively unlimited scalability, it seems like the value proposition of cloud LLM gets ever shakier. * Many businesses don't need frontier level intelligence anyway. * It's completely stateless. If your local LLM machine catches fire? Nothing was lost. Buy another.

Yeah. Local agent sessions are not backed up in the cloud. And uptime is better with local models.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#168

Post by vanviegen »

suprjami wrote: post #63
jedbrooke wrote: post #50

I feel like the current “reasoning” that LLMs are doing has got to be a dead end eventually. Every time I have to read another answer with “but wait” and “Actually,” as they “reason” their way to a (sometimes) better answer, I feel like there’s got to be a way to just shortcut to the actual correct answer instead of burning all these token going in circles mimicking actual thought

So-called "caveman" thinking attempts to address this. The important part of "actually wait, I really need to XYZ" is just "XYZ". The model can attend to just "do XYZ" and produce almost the same vector modifications as full verbose "reasoning".

I don't think that's true. If a context contains a statement followed by something opposing that statement, that will confuse the model. So "actually wait, I really need to" is there to signal that the previous thinking may be flawed and that what follows is a new attempt.

It's good to remember that LLMs have no more state then what they can derive from the context up til any point. So if that context is hard to interpret, that will reduce effectiveness.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#169

Post by hellajack3d »

I forked llama.cpp and added some crude mechanism to keep exactly this behavior under control - essentially guiding the reasoning process by injecting text strategically at specific thresholds. This was mainly put together to rein in Qwen3.6-27B, but I'd imagine 3.8 would react similarly.

Fork can be found here - https://github.com/laurencehardman/llama-mindcontrol/tree/ma...

Of course hacks like this are not perfect and may degrade performance slightly due to injected text pushing the model slightly out-of-distribution, so the string constants need to be chosen carefully - Qwen3.5's technical whitepaper does provide some guidance in this regard. The mechanism is absolutely more of a hack than a feature, and i'd imagine will be made redundant once llama.cpp supports more appropriate reasoning controls - but for now, i've found it pretty useful.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#170

Post by rene-veerman »

mattmaroon wrote: post #151
redrove wrote: post #145

Earlier quoted context omitted.

Thanks for the very detailed answer! Would you say ChatGPT does everything you need today or do you still see some gaps? i.e. things you think it should be able to do but currently doesn’t, or things that still take too much effort on your part to setup chatgpt to do it.

Huge gaps. I expect the tooling to get better for non-programmers. Codex and Cowork are great, but you still feel like you’re trying to hammer the square peg through the circle hole often when using it for non-programming tasks. The AI is good enough to do a lot of tasks but the tooling just isn’t caught up to it yet. I’d say it’s freed up ten hours a week of my time. And that’ll only improve.

how'd you get it to run? not even upgrading ollama could get it installed on my end :(
Post reply on HN