Qwen 3.8 27B is excellent, but it defaults to overthinking things • Hacker News Board

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#391

Post by roosterIllusi0n » Tue, Aug 18, 2026, 8:27 AM UTC

dofm wrote: post #305

CMay wrote: post #171

Earlier quoted context omitted.

For me that moment was Gemma 4 12B QAT. You're not suddenly going to start throwing your hardest programming problems at Gemma 4 12B QAT, it is still 15B parameters less. It's more that, aside from pelican art which isn't what local models are for, I didn't see anything on Simon's post that it couldn't assist with or largely succeed at. It can run 80-100t/s on a laptop, can understand images natively and do bounding…

> It's only 7GB including the mmproj or 8GB with MTP. Even more impressively it doesn't have a separate mmproj at all — it is fully integrated, and the vision encoder doesn't speak words into the LLM, as it were —- it is directly integrated into the model's weights. I have banged on about this model here enough but I really agree that Gemma 4 12B is a candidate for the most impressive LLM of the year. It is remarkabl…

My issue with Gemma 4 is that any task fails to complete after any compaction event. It often ends up in a loop that keeps compacting and showing the same compaction output. Qwen3.8-27B-IQ4_XS was a massive improvement. It's tasks survive compaction and actually get completed. I switched to Qwen3.8-27B-UD-Q3_K_XL for better performance and its working just as well.

Gemma4 screwed up a proxmox install I had. I booted to a SystemRescue install and tried to get gemma4 to fix it. It just could not do it and kept having issues where it dropped a linux command into the local powershell because it did not ssh into systemRescue or killed the ssh connection somehow so the text landed on the wrong system.

I told qwen3.8 to investigate fixing the partition. It said information was lost, but displayed enough info that it was easy to tell it was right. I told it to install fresh proxmox and gave a short rundown on settings and partition sizes I wanted. It made a plan and told me I had to manually installed proxmox by booting the iso. I responded with something like "there are other ways to install promox without human interaction so use one of those". That was it. I woke up to the system having booted to a new proxmox install with my previous ssh keys restored and my existing zfs pool already mounted.

I don't see how any model that is limited to a single context window in a single session would be viable for coding. I want something that can manage the entire project and not just individual files or inline suggestions. I need to be able to feed it all the info I would use to make coding decisions and then have it at least make a working project that it can launch and test successfully. You want it to ask as many questions up front to enable continuous work without stopping for human input.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#392

Post by 59nadir » Tue, Aug 18, 2026, 9:05 AM UTC

CMay wrote: post #171

chvid wrote: post #66

“The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.” I think that should be the blinking headline - this shows what can be done with consumer hardware.

For me that moment was Gemma 4 12B QAT. You're not suddenly going to start throwing your hardest programming problems at Gemma 4 12B QAT, it is still 15B parameters less. It's more that, aside from pelican art which isn't what local models are for, I didn't see anything on Simon's post that it couldn't assist with or largely succeed at. It can run 80-100t/s on a laptop, can understand images natively and do bounding…

I've found that the Gemma series of models are made for someone entirely different than myself. They fail at even the most basic questions I throw at them, like 12B just now failed at answering how `XGrabKey` from Xlib is used. It hallucinated the entire API and made up an entire flow of code based on it, for no particular reason. It could've even decided to research this via web search because I have a tool specifically set up for that, but it "chose" not to, relying instead on completely made up information.

This isn't an isolated incident, really, I find myself always having these issues with the Gemma series. I'm sure they can do useful things for someone else, but for the things I want to use LLMs for (very small code generation, quick questions, code review) they always seem to disappoint me. I'm sure it's because of the stuff that I do and use, but it's a very consistent red thread with these models for me.

Edit:

The same question for Qwen3.6-35B-A3B produces a pretty concise and correct answer that would be useful to the questioner, without even going to the web. I don't know what Gemma models are trained on, but it's not the stuff that's relevant to me.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#393

Post by roosterIllusi0n » Tue, Aug 18, 2026, 9:08 AM UTC

DanielHB wrote: post #222

CMay wrote: post #218

Earlier quoted context omitted.

Google provided incorrect settings and an imperfect template. Unsloth modified the template and then finetuned their own version of the model to optimize for some benchmarks as a means of validating quants. Google and Llama.cpp then adopt template changes by default, so anyone downloading the new model or even using the original model will now automatically be using it incorrectly. Llama.cpp also uses the same infere…

I am very much a beginner to local LLM stuff and I find it incredibly hard to figure out how to run models optimally with the correct settings for my hardware. The number of different variations of the same model and how each quant work is super confusing as well. When I tried to run llama.cpp directly I was getting max 9tk/s on qwen3.5-9B, then I tried LM Studio with the same model and got 77tk/s. I haven't figured…

I have had luck telling the free chatgpt my graphics card brand/vram and asking it to recommend latest qwen3.8 or gemma4 model variants. Then I pasted in my server command and ask it to optimize it. I also pasted in the token per second logs to get further tweaks. If you paste the token per second info log info back to chatgpt, you can iterate with the free chatgpt to get better settings.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#394

Post by throwawayffffas » Tue, Aug 18, 2026, 10:03 AM UTC

jedbrooke wrote: post #50

I feel like the current “reasoning” that LLMs are doing has got to be a dead end eventually. Every time I have to read another answer with “but wait” and “Actually,” as they “reason” their way to a (sometimes) better answer, I feel like there’s got to be a way to just shortcut to the actual correct answer instead of burning all these token going in circles mimicking actual thought

I think you should avoid reading the thinking blocks unless you spot errors in the output.

I am very confident the reason we get all these second guessing and "but wait" and "actually" is they train them on collapsed corrected sessions. i.e they take sessions that look like this:

  user: Do x.
  agent: the user wants me to do x. I think I need to do a and b first.
  agent: does a.
  agent: does b.
  user: No no no doing a was wrong you should do c before b.
  agent: undoes a. does c.
  agent: does x

And they turn it to a session where the user correction shows up in the thinking. i.e

  user: do x.
  agent: the user wants me to do x. I think I need to do a and b first.
  agent: but wait maybe I should do c instead of a
  agent: does c
  agent: does b
  agent: does x

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#395

Post by DanielHB » Tue, Aug 18, 2026, 11:06 AM UTC

eru wrote: post #379

DanielHB wrote: post #266

Earlier quoted context omitted.

Like I said, just feels I have from the consumer-hardware space. For several generations of GPU now most improvements come from packing more transistors into a larger die than packing more transistors closer to each other. GPUs have been getting physically bigger with huge heatsinks and fans to support those bigger dies power consumption. Just compare the TDPs: 2020 RTX 3090: 350W 2022 RTX 4090: 450W 2025 RTX 5090: 5…

> GPUs have been getting physically bigger with huge heatsinks and fans to support those bigger dies power consumption. Just compare the TDPs: You can also look at what's been happening in mobile and especially with Apple's integrated processors. They are more power constrained, so people worried more about power there.

I believe the power gains from apple stuff mostly come from CPU and memory interconnect, unrelated to consumer-grade PC GPUs. Server-grade GPUs have different memory interconnect architecture which I assume already has similar power efficiency gains.

I think raw flops per watt come mostly from fab process, not architecture. This was my original point, fab process is not getting better at a linear (much less exponential) scale anymore.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#396

Post by noman-land » Tue, Aug 18, 2026, 12:24 PM UTC

busfahrer wrote: post #238

paulbjensen wrote: post #237

I just used it on a Apple M4 MacBook Pro with 48GB RAM with llama.cpp and Pi to help diagnose an infinite looping request in a React Server component on a Next.js application. After about 10+ hours of digging, it has apparently found a bug in the Next.js framework, with an example app that replicates the bug, and a fix for now to disable prefetch in the Link component. I had in my prompt asked it to discover the root…

I am eyeing one of these specifically for this use case, could you please post roughly what kind of tokens per second numbers you get for text generation for this 27B model? edit: and which quant you are using, please :-)

Using the 4bit quant on an M1 64GB I'm getting ~65 tps for prompt processing and ~11 tps token generation using oMLX to serve the models and pi as a harness.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#397

Post by shafyy » Tue, Aug 18, 2026, 1:02 PM UTC

piva00 wrote: post #209

shafyy wrote: post #200

Earlier quoted context omitted.

Can it? I feel like I instantly recognize if I am chatting with an LLM or a human

I was quite surprised on how difficult it is to tell when chatting with an uncensored LLM a friend is running (it's too big to run on any of my computers but he got some B200s). You can input your own "system prompt" to make it behave like a normal internet user and the prose writes very similarly to internet comments with none of the LLMisms from ChatGPT, Claude, Grok, etc.

Yes, I'm sure it's possible to get LLMs to respond more like an average human would.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#398

Post by shafyy » Tue, Aug 18, 2026, 1:03 PM UTC

coldtea wrote: post #275

shafyy wrote: post #200

Earlier quoted context omitted.

Can it? I feel like I instantly recognize if I am chatting with an LLM or a human

"Feel" is doing a lot of work here. You can recognize 70% of those (true positive rate) and still have a false negative rate of 30%, while thinking you got 100% of the AI ones! The problem is that you'd be oblivious to those you don't recognize.

70%? Are we just making up numbers now?

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#399

Post by SwellJoe » Wed, Aug 19, 2026, 1:26 AM UTC

SwellJoe wrote: post #61

syntaxing wrote: post #32

Earlier quoted context omitted.

What speed do you get on this setup? Im tempted to use the same GPU.

It varies widely based on a bunch of factors. With this specific model at 8-bit quantization and MTP, it starts out at about 25 t/s for basic chat, but for agentic tasks with long context it slows way down to something like 12-15. I don't see a big difference in token rates based on any config changes I have tried, or going to the smaller 6-bit quantization, so far, though I haven't spent a ton of time on experimenti…

Update on this: When I enable tensor parallelism in llama.cpp, I see 25-33 t/s. With reasoning effort set to medium, Qwen 3.8 finished the same task that previously took 11 hours in a little over three hours, which is still more than three times what most of the large models required including Opus 4.8, and nine times what GPT 5.5 (the fastest of the models I've used) needed for a similar task. So, it's still not fast enough for comfort, but it's much faster than the first run. And, I guess, faster than writing the code myself.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#400

Post by syntaxing » Wed, Aug 19, 2026, 1:33 AM UTC

SwellJoe wrote: post #399

SwellJoe wrote: post #61

Earlier quoted context omitted.

It varies widely based on a bunch of factors. With this specific model at 8-bit quantization and MTP, it starts out at about 25 t/s for basic chat, but for agentic tasks with long context it slows way down to something like 12-15. I don't see a big difference in token rates based on any config changes I have tried, or going to the smaller 6-bit quantization, so far, though I haven't spent a ton of time on experimenti…

Update on this: When I enable tensor parallelism in llama.cpp, I see 25-33 t/s. With reasoning effort set to medium, Qwen 3.8 finished the same task that previously took 11 hours in a little over three hours, which is still more than three times what most of the large models required including Opus 4.8, and nine times what GPT 5.5 (the fastest of the models I've used) needed for a similar task. So, it's still not fas…

Thanks for the update! I wonder if a Blackwell GPU would be noticeably faster. Which vendor did you end up using? I want to get a gigabyte one but thats be OOS for months.