Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B is excellent, but it defaults to overthinking things

simonwillison.net

Post reply on HN
Print view View original

191–200 of 411 posts

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#191

Post by picture »

jongjong wrote: post #72
jatora wrote: post #38

All current era models overthink as it's a product of their RL incentives (or distillation of models with them...) From my reading of the Fable 5 and Opus 5 System cards, my reconstruction is something like: Finish the task → make externally observable evidence that it is finished → check your own work → fix problems → don't stop prematurely → satisfy the evaluator comprehensively. That is fantastic for SWE benchmark…

For coding, this is very interesting because the same incentives were present for humans before AI. Tech companies which had a culture of rewarding complexity would see huge Pull Requests and a lot of unnecessary complexity. I've worked in companies which would require a thousand lines of code to implement a feature which would require only a hundred or so lines at a different company. The shorter one was more reliab…

Well, it's good that there's some competition within the space. More compact/correct/elegant code is simply better code, and people will catch on to that eventually. I think there is a stronger incentive to gain market share and higher profits vs. naively skimming a small additional margin by sandbagging efficiency/performance

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#192

Post by mdp2021 »

Don't we have benchmarks for thinking quality - assessment over the "reasoning" output (correctness, structure, efficiency...)? We definitely should.

And before the benchmark of the finished LLM, it would be interesting to consider the techniques used by LLM producers during training to optimize the "think" chunk quality. I cannot remember any good articles about it now.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#193

Post by podocarp »

CMay wrote: post #171
chvid wrote: post #66

“The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.” I think that should be the blinking headline - this shows what can be done with consumer hardware.

For me that moment was Gemma 4 12B QAT. You're not suddenly going to start throwing your hardest programming problems at Gemma 4 12B QAT, it is still 15B parameters less. It's more that, aside from pelican art which isn't what local models are for, I didn't see anything on Simon's post that it couldn't assist with or largely succeed at. It can run 80-100t/s on a laptop, can understand images natively and do bounding…

Out of the loop here. What did Google and unsloth and llama do to mutilate Gemma? I can understand Google shenanigans but llama and gunsmith is kind of surprising.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#194

Post by wejick »

chvid wrote: post #66

“The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.” I think that should be the blinking headline - this shows what can be done with consumer hardware.

In this kind of moment, I really wished hardware manufacturing and demand situation is in much state. Imagine this can be accessible by everyday people with only 6 months hardware market gap. The societal impact would be much bigger.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#195

Post by Balinares »

Reproducing the post's money quote here because it's absolutely the crux of why the Qwen 3.8 release is seismic IMO:

"The models at this size continue to get better at an impressive rate. We don’t need to spend half a million dollars on datacenter-class hardware just to run a competent model."

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#196

Post by LinXitoW »

SahAssar wrote: post #130
ipdashc wrote: post #114

Earlier quoted context omitted.

I mean, I might be missing something, but isn't part of the idea with cloud infra that you can scale down as well? Large orgs with significant demand might go out and buy local LLM hardware, but most businesses probably don't want to bother dropping $2k on a box with a beefy GPU and would rather just pay the lowest subscription tier so their employees can occasionally make queries. Plus, you know, the whole economies…

I'm pretty sure a small-to-medium org has plenty of things that could be queued/scheduled to run when there is downtime. It requires some planning and thought though and I don't think most orgs are there yet.

Yeah, but again, why would you? When it comes to open weight models, there's a good amount of competition, so you already get a really good price, without any optimization or anything.

Seriously, you can do years of Deepseek inference for the hardware to run just 1 or 2 requests against a slower, dumbed down model on your own hardware.

It makes no sense to buy hardware right now, when the price is completely disconnected from any material reality. It's much better to use the cloud providers VC funding by using their cheap as F offering. Either AI becomes less useful, or hardware costs come down. Either way, you'll be in a better position in 3 years than you are today.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#197

Post by digikata »

Qwen3.6 27B is very usable, and dialing back thinking modes woth Qwen3.8 bring it close, but 3.8 stills feels slower. Unknown to me if the results are qualitatively better or worse overall - with the heavier thinking 3.8 felt worse in terms of coding tasks, but I think Im comparing a newly released model to one that has had a lot of harness tuning. 3.6 27B was easily a daily driver with only an occasional need to pop up to larger models for planning.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#198

Post by CamouflagedKiwi »

bakraman wrote: post #146
simonw wrote: post #83

Earlier quoted context omitted.

It should just about be usable in 32GB.

RAM is never the issue, it's always the compute power

It's absolutely not for these models. There are plenty of consumer GPUs out there with 8 or 12GB VRAM - they are comparatively very fast at inference but just aren't big enough to run lots of the models you want. Also context management is a massive pain.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#200

Post by shafyy »

coldtea wrote: post #179
bitwize wrote: post #141

Earlier quoted context omitted.

Indeed. LLMs resemble human intelligence in more or less the same way that the output of the TI-99/4A speech synthesizer resembles a human voice.

Not if an LLM over chat can fool most people they're talking to a human (which it can), where the TI-99 speech synthesizer voice absolutely can not.

Can it? I feel like I instantly recognize if I am chatting with an LLM or a human
Post reply on HN