Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B is excellent, but it defaults to overthinking things

simonwillison.net

Post reply on HN
Print view View original

141–150 of 411 posts

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#141

Post by bitwize »

a_e_k wrote: post #115
chvid wrote: post #66

“The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.” I think that should be the blinking headline - this shows what can be done with consumer hardware.

Like the old proverb: "The marvel is not that the bear dances well, but that the bear dances at all."

Indeed. LLMs resemble human intelligence in more or less the same way that the output of the TI-99/4A speech synthesizer resembles a human voice.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#142

Post by XCSme »

My comparison of its reasoning efforts[0] seems to show that it only really supports 3 modes: none, low, xhigh.

Low and medium are basically the same.

Also, the electricity it costs to run on a 3090 is not negligible, so that it's cheaper to use Luna high via API than Qwen 3.8 27b locally, hardware costs excluding.

[0]: https://aibenchy.com/compare/qwen-qwen3-8-27b-high/qwen-qwen...

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#143

Post by CapsAdmin »

javchz wrote: post #7

I wonder if this can be fixed with LORAs.

This morning I tried experimenting with this ThinkingCap lora I found someone made for 3.6 https://huggingface.co/signsur4739379373/Qwen3.6-27B-Thinkin...

ThinkingCap is a 3.6 27b finetune that claims to halve thinking tokens while maintaining the same output quality. I've used the model a lot and I'd say it holds up. Since 3.6 has the same architecture as 3.8, the lora can be applied.

With the prompt "create a fancy circle in html", these are the results for xhigh, medium, low and xhigh + thinkingcap lora

https://gist.github.com/CapsAdmin/b0ea64006f942c5a96a56dba78...

(Note that the gists are bloated because they contain the full chat and launch params in text/plain script tags for transparency)

I'd say xhigh looks a little better than xhigh + lora, but the lora variant has 40% less thinking tokens. Both seemed to take the same approach with adding random details that weren't explicitly specified.

Medium and low (no lora) are close to each other but are much simpler results.

This is just me testing a single turn. I haven't tested this on multi turns and whatnot, but I thought the result was interesting enough to share anyway.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#144

Post by eru »

ZaoLahma wrote: post #116
chvid wrote: post #66

“The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.” I think that should be the blinking headline - this shows what can be done with consumer hardware.

Full agree. I until very recently thought AI tools of today were limited to prohibitively expensive high end hardware hosted in data centers. I was surprised and amazed to get "decent" (with the expectations set right / low) coding performance out of Qwen3.5-9B on a decidedly medium end Radeon 9070 paired with a 5700x3d and 32GB of DDR4 RAM. We can finally reason with and "talk" to our hardware.

Yes, and we are still pretty early: AI is still advancing at breakneck speeds, and hardware is too.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#145

Post by redrove »

mattmaroon wrote: post #133
redrove wrote: post #129

Earlier quoted context omitted.

I would also be interested in what you use AI for. Is it marketing? bureaucracy?

Yes. Bureaucracy for sure when I throw festivals. ChatGPT pulls permits for me. (I think that uses a lot of tokens because of the browser control?) Manages the admin side along with some tools I built in Lovable via mcp servers. Marketing definitely. My food truck side is relatively high volume and it manages my kitchen and warehouse side, basically generating all of the instructions my employees follow, managing and…

Thanks for the very detailed answer!

Would you say ChatGPT does everything you need today or do you still see some gaps? i.e. things you think it should be able to do but currently doesn’t, or things that still take too much effort on your part to setup chatgpt to do it.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#147

Post by redrove »

NhanH wrote: post #139
redrove wrote: post #128

Earlier quoted context omitted.

I don’t disagree but saying “cloud will change in the next 8-20y-ish” is a bit of a non-argument, you’re not really stating any thesis to speak of; Change is a given over that time frame.

Ah it was not explicit enough. They will die, for some definitions of death -- I don't think they disappear, but they should be a niche, rather than the dominant doctrine.

I’m not so sure I agree, given the overwhelming concentration of capital and regulatory capture they have, I just don’t see them going anywhere; becoming more niche rather than even more of a standard is “going away” to a certain extent as far as I can see.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#148

Post by freehorse »

bitwize wrote: post #141
a_e_k wrote: post #115

Earlier quoted context omitted.

Like the old proverb: "The marvel is not that the bear dances well, but that the bear dances at all."

Indeed. LLMs resemble human intelligence in more or less the same way that the output of the TI-99/4A speech synthesizer resembles a human voice.

I also believed that, but seeing qwen 27b overengineering solutions in a bit too familiar way in the article, I started doubting that.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#150

Post by monster_truck »

XCSme wrote: post #142

My comparison of its reasoning efforts[0] seems to show that it only really supports 3 modes: none, low, xhigh. Low and medium are basically the same. Also, the electricity it costs to run on a 3090 is not negligible, so that it's cheaper to use Luna high via API than Qwen 3.8 27b locally, hardware costs excluding. [0]: https://aibenchy.com/compare/qwen-qwen3-8-27b-high/qwen-qwen...

$0.286/kWh is a ridiculous amount of money to pay for power. That's more than double the regional residental average here!

If I ever found myself in this situation I would much rather just rent cards from hotasile and run open models instead of giving OAI money and playing reset bingo

Post reply on HN