Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B is excellent, but it defaults to overthinking things

simonwillison.net

Post reply on HN
Print view View original

211–220 of 411 posts

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#211

Post by petu »

romeinaday wrote: post #190

Can you run this on a 36GB MacBook Pro (M3 Pro)? What would be a good setup? for coding mainly

You need ~24-26GB for basic setup (17-19GB model + 128K 8 bit context), so you can, but not much memory would be left for doing anything else on that machine. And even then it would run at like 5-10 t/s (due to relatively low memory bandwidth of M3 Pro) and slow prompt processing (couple hundreds of t/s?)

If they end up releasing updated 35B-A3B variant, then it would be much more interesting in generation speed (~50 t/s)

For inference engine/server you have two (main) choices: llama.cpp for platform-agnostic, MLX for Apple-only. They will spin up OpenAI-compatible local server, and you point your agent harness to it.

For llama.cpp this should be reasonable (maybe shrink context to 128K) starting point: https://x.com/ggerganov/status/2088312671196082312

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#212

Post by zmmmmm »

Looking at the example where he asked for an SVG of a circle and it spent ages and drew a spectacular animated SVG with shading and a rotating arrow.

It's honestly a bit concerning, I'm seeing this across the board (Opus 5, looking at you). Nearly all the AI models are doing more than they are asked for. I assume this is helping them win benchmarks but I see it as almost as misaligned as deliberately doing the wrong thing altogether. This is how you end up with your AI model hacking into someone else's server or backdooring your code so it will have future access to debug things.

I think we need somehow to address this in the benchmarks before before things get even worse.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#213

Post by coldtea »

zahlman wrote: post #210
coldtea wrote: post #179

Earlier quoted context omitted.

Not if an LLM over chat can fool most people they're talking to a human (which it can), where the TI-99 speech synthesizer voice absolutely can not.

> Not if an LLM over chat can fool most people they're talking to a human (which it can) I keep hearing this claim, and yet I keep seeing LLM output which is trivially distinguished from human writing. I really can't understand how this gap persists; but then, there seem to have been at least some people who couldn't sniff out ELIZA, back in the day, too.

>I keep hearing this claim, and yet I keep seeing LLM output which is trivially distinguished from human writing.

That's mostly true for longer LLM output with all the sycophancy / LinkedIn bias thrown in.

Make it casual conversation or comments, and give it instructions on appearing casual, or even better kill the censoring and fixed-prompt (with an open model), and it's orders of magnitude more difficult, unless if you suspect it and try specifically tailored prompts to sniff it.

There's no shortage of people obliviously discussing with AI bots in comment sections.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#215

Post by zahlman »

coldtea wrote: post #213
zahlman wrote: post #210

Earlier quoted context omitted.

> Not if an LLM over chat can fool most people they're talking to a human (which it can) I keep hearing this claim, and yet I keep seeing LLM output which is trivially distinguished from human writing. I really can't understand how this gap persists; but then, there seem to have been at least some people who couldn't sniff out ELIZA, back in the day, too.

> I keep hearing this claim, and yet I keep seeing LLM output which is trivially distinguished from human writing. That's mostly true for longer LLM output with all the sycophancy / LinkedIn bias thrown in. Make it casual conversation or comments, and give it instructions on appearing casual, or even better kill the censoring and fixed-prompt (with an open model), and it's orders of magnitude more difficult, unless i…

At the bottom of this very submission are a bunch of dead comments that are very obviously LLM-generated.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#216

Post by petu »

madduci wrote: post #87
chvid wrote: post #66

“The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.” I think that should be the blinking headline - this shows what can be done with consumer hardware.

Tried yesterday on my own laptop (a UltraCore 7 255H without dedicated GPU,with 32 GB RAM), it wasn't even starting thinking, even on a small context window (65k)

What was your prompt length? It's possible it was just processing it and it's likely not fast on your setup.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#218

Post by CMay »

podocarp wrote: post #193
CMay wrote: post #171

Earlier quoted context omitted.

For me that moment was Gemma 4 12B QAT. You're not suddenly going to start throwing your hardest programming problems at Gemma 4 12B QAT, it is still 15B parameters less. It's more that, aside from pelican art which isn't what local models are for, I didn't see anything on Simon's post that it couldn't assist with or largely succeed at. It can run 80-100t/s on a laptop, can understand images natively and do bounding…

Out of the loop here. What did Google and unsloth and llama do to mutilate Gemma? I can understand Google shenanigans but llama and gunsmith is kind of surprising.

Google provided incorrect settings and an imperfect template.

Unsloth modified the template and then finetuned their own version of the model to optimize for some benchmarks as a means of validating quants.

Google and Llama.cpp then adopt template changes by default, so anyone downloading the new model or even using the original model will now automatically be using it incorrectly.

Llama.cpp also uses the same inference setting defaults regardless which version of the model you use and some settings are simply defaults it uses for all models.

Then even if you account for all of these, you have to be using Gemma 4 itself correctly, which many people do not.

All of these little changes and inconsistencies hurt some of the model's original capabilities. Even if you go directly to Google's repo and download the full float 16 weights with the template they have there now, you cannot simply assume you're getting the best results.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#219

Post by entrope »

niek_pas wrote: post #205
nsingh2 wrote: post #64

Earlier quoted context omitted.

One line of evolution seems to be toward some form of latent-space reasoning, as in [1]. Natural language seems like a relatively low-bandwidth channel for intermediate reasoning. [1] https://github.com/sapientinc/HRM-Text

What does ‘latent’ mean in this context?

"Latent" means it (the space) is there all the time but usually not exposed or apparent. Latent space is a high-dimensional space that is not legible to humans, so normally reasoning gets constrained to progress in tokens instead so that it can be evaluated.

I think https://arxiv.org/abs/2412.06769 is a good description of the premise of reasoning in latent space, although https://arxiv.org/abs/2604.15726 argues it's already what really happens.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#220

Post by XCSme »

sgt wrote: post #184
XCSme wrote: post #142

My comparison of its reasoning efforts[0] seems to show that it only really supports 3 modes: none, low, xhigh. Low and medium are basically the same. Also, the electricity it costs to run on a 3090 is not negligible, so that it's cheaper to use Luna high via API than Qwen 3.8 27b locally, hardware costs excluding. [0]: https://aibenchy.com/compare/qwen-qwen3-8-27b-high/qwen-qwen...

Btw, unrelated, but this is the kind of Vibe Coded AI slop design I see a lot these days. Every single thing on this page has a different color, formatting, and it's just painful to look at.

Thanks for the feedback!

That might be my fault, I am not a designer, and I asked for most UI decisions.

I tried to use colors to diferentiate models, the site is very data dense and it's hard to make everything readable.

I've spent hundreds of hours building it, not sure if I would call it slop, but I just suck at design, lol

Any suggestions on how to improve it?

Post reply on HN