Live data from Hacker News

Qwen 3.8 27B is excellent, but it defaults to overthinking things

simonwillison.net

401–410 of 411 posts

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#401

Earlier quoted context omitted.

Update on this: When I enable tensor parallelism in llama.cpp, I see 25-33 t/s. With reasoning effort set to medium, Qwen 3.8 finished the same task that previously took 11 hours in a little over three hours, which is still more than three times what most of the large models required including Opus 4.8, and nine times what GPT 5.5 (the fastest of the models I've used) needed for a similar task. So, it's still not fas…

Thanks for the update! I wonder if a Blackwell GPU would be noticeably faster. Which vendor did you end up using? I want to get a gigabyte one but thats be OOS for months.

The Blackwell and Strix Halo will be similar to each other and much slower than the number I'm getting on the dual V620 setup (I see about 10-15 t/s on my Strix Halo with this model at 8-bit quantization depending on context). Prefill is generally quite a bit faster on the DGX Spark and token generation slightly faster on the Strix Halo, as I understand it. But, there are better software efficiency improvements for the Spark line.

This model is far from usable on current AMD or Nvidia 128GB AI machines, IMHO, they just don't have the memory bandwidth, especially since it chews so many tokens for any task. If you want to run this specific model, two (or more) 32GB GPUs with decent memory bandwidth is the right way to do it. It doesn't benefit from the larger memory of the Strix Halo. There's enough room for full context and 8-bit quantized model in 64GB. But, it's really a terrible time to buy hardware. MoE models are a much better fir for the Spark and Strix Halo; you can run Laguna S2.1 (slowly) or one of the Qwen 3.6 MoE fine-tunes (pretty quick). Ling 3.0 Flash also looks promising. Nemotron 3.5 Lightning in the MXFP4 quantization absolutely flies on the Strix Halo at 65-80 t/s, but it's dumb. But, all of those are weaker than Qwen 3.8 27B for coding.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#402
post #182

Earlier quoted context omitted.

Ridiculous? Wow. I'm paying ~$0.4/kWh in western Europe...

Paying around ~$0.25-0.3 per kWh here in South Africa. But you can also put up a lot of solar panels and lower that effectively.

Wow. Those are about EV quick charge prices here in .fi. Average residential is maybe 0.06 €/kWh (plus transfer and subscription).

https://sahkovertaaja.fi/porssisahkon-keskihinta/>

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#403
post #66

“The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.” I think that should be the blinking headline - this shows what can be done with consumer hardware.

I totally agree. Is a local model running on your laptop going to outperform the latest frontier model? No, but that's not the point. Many of the use cases folks have can be done well with these newer smaller models. What amazes me is that these keep getting better with existing hardware you have. It's been fun to benchmark and test as these keep coming out.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#404
post #355

Can you run this on a 36GB MacBook Pro (M3 Pro)? What would be a good setup? for coding mainly

i have this exact same machine and 3.8-27b runs well, albeit slow at about 6 t/s. i have it complete tasks in the background as i work on other things. for comparison, my 3090 runs it at about 60 t/s.

What are you using to run the model and as coding harness?

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#405
post #211

Can you run this on a 36GB MacBook Pro (M3 Pro)? What would be a good setup? for coding mainly

You need ~24-26GB for basic setup (17-19GB model + 128K 8 bit context), so you can, but not much memory would be left for doing anything else on that machine. And even then it would run at like 5-10 t/s (due to relatively low memory bandwidth of M3 Pro) and slow prompt processing (couple hundreds of t/s?) If they end up releasing updated 35B-A3B variant, then it would be much more interesting in generation speed (~50…

So you can use llama.cpp and point a harness to it? What harness do you use?

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#406
post #355

Earlier quoted context omitted.

i have this exact same machine and 3.8-27b runs well, albeit slow at about 6 t/s. i have it complete tasks in the background as i work on other things. for comparison, my 3090 runs it at about 60 t/s.

What are you using to run the model and as coding harness?

llama.cpp + pi.dev

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#408

Earlier quoted context omitted.

Oh I’m just using ChatGPT, I haven’t dives into getting anything to run locally yet. That’s on the off season project list.

i see. i'm eager to use the self-run versions of real code-assisting AIs you see. ah well, patience. more patience. .

Me too.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#409
From my experience, I suspect you'll need to be able to use both thinking modes (medium and xhigh).

I did set the default at medium, but the xhigh still seems to be a big part of its benefits.

I found it really easy to chat with it and instantly see flaws in its thinking when using the medium level of reasoning... in a way that had me consider that the frontier models reasoning abstractions (both at the harness level, e.g., claude code, codex) AND in the server-side obfuscation) that I found refreshing, because it made it easy for me to step in and precisely identify the failures of reasoning that the frontier models were getting stuck on, and because of the 'black box' hiddenness of their reasoning, it made it harder for me to diagnose.

So, I suppose I'm saying 'there is a time and place for each'.

Here is my setup: https://williamcallahan.com/blog/qwen-3-8-27b-is-a-great-ope...

Post reply on HN