Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B is excellent, but it defaults to overthinking things

simonwillison.net

Post reply on HN
Print view View original

181–190 of 411 posts

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#181

Post by mdp2021 »

madduci wrote: post #87
chvid wrote: post #66

“The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.” I think that should be the blinking headline - this shows what can be done with consumer hardware.

Tried yesterday on my own laptop (a UltraCore 7 255H without dedicated GPU,with 32 GB RAM), it wasn't even starting thinking, even on a small context window (65k)

Have you tried with different amounts for the "reasoning_effort (xhigh|medium|low)" parameter?

Or the " | | " tags: apart from this template detail, it is not immediately clear if reasoning_effort is deterministic (API) or is prompt engineering.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#182

Post by sgt »

AgentMatt wrote: post #158
monster_truck wrote: post #150

Earlier quoted context omitted.

$0.286/kWh is a ridiculous amount of money to pay for power. That's more than double the regional residental average here! If I ever found myself in this situation I would much rather just rent cards from hotasile and run open models instead of giving OAI money and playing reset bingo

Ridiculous? Wow. I'm paying ~$0.4/kWh in western Europe...

Paying around ~$0.25-0.3 per kWh here in South Africa. But you can also put up a lot of solar panels and lower that effectively.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#183

Post by sgt »

dexterlagan wrote: post #132

I run mine on an M5 Max with just 48GB of (V)RAM, and it fits nearly twice in Q4. Works perfectly. I'm kinda glad I didn't spend the extra $2400 to get 128. We don't really need more... and that's a good thing (tm). God knows I thought about it in store. But I thought... maybe this year will be the year of the local model? Maybe soon we won't need that much RAM? I was right. The fact that it runs at 15tk/s in power s…

For day to day LLM experimentation (and even some business use cases), I'd say Apple Silicon would be first choice for me.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#184

Post by sgt »

XCSme wrote: post #142

My comparison of its reasoning efforts[0] seems to show that it only really supports 3 modes: none, low, xhigh. Low and medium are basically the same. Also, the electricity it costs to run on a 3090 is not negligible, so that it's cheaper to use Luna high via API than Qwen 3.8 27b locally, hardware costs excluding. [0]: https://aibenchy.com/compare/qwen-qwen3-8-27b-high/qwen-qwen...

Btw, unrelated, but this is the kind of Vibe Coded AI slop design I see a lot these days. Every single thing on this page has a different color, formatting, and it's just painful to look at.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#185

Post by runtime_lens »

CMay wrote: post #171
chvid wrote: post #66

“The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.” I think that should be the blinking headline - this shows what can be done with consumer hardware.

For me that moment was Gemma 4 12B QAT. You're not suddenly going to start throwing your hardest programming problems at Gemma 4 12B QAT, it is still 15B parameters less. It's more that, aside from pelican art which isn't what local models are for, I didn't see anything on Simon's post that it couldn't assist with or largely succeed at. It can run 80-100t/s on a laptop, can understand images natively and do bounding…

[flagged]

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#186

Post by cyberrock »

noduerme wrote: post #131
aphroz wrote: post #91

Earlier quoted context omitted.

I think not much can run without a dedicated GPU

What's the story with Mac laptops? Worth a try?

Dense ones like this are more bandwidth-hungry, so you want to try MoE ones like Qwen3.6-35B-A3B (35 Billion params but only 3 Billion Active) or Gemma 4. Unfortunately it seems like we might not be getting a 3.8 MoE.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#187

Post by vorticalbox »

dexterlagan wrote: post #132

I run mine on an M5 Max with just 48GB of (V)RAM, and it fits nearly twice in Q4. Works perfectly. I'm kinda glad I didn't spend the extra $2400 to get 128. We don't really need more... and that's a good thing (tm). God knows I thought about it in store. But I thought... maybe this year will be the year of the local model? Maybe soon we won't need that much RAM? I was right. The fact that it runs at 15tk/s in power s…

Have you looked at using oMLX?

https://omlx.ai/

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#188

Post by Balinares »

Worth noting that the default GGUF template sets the reasoning to xhigh. You can use the Froggeric template to set reasoning to medium instead: https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

Worth noting as well that the weights come with an MTP layer that seems particularly accurate, to the point it can give you up to 6-8 correctly predicted tokens consistently enough to be useful. Which obviously boosts its speed enormously.

I find it difficult to believe how good this model is. It feels like it's lagging heavyweight frontier models by less than a year, and it runs on your PC.

Post reply on HN