Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B is excellent, but it defaults to overthinking things

simonwillison.net

Post reply on HN
Print view View original

291–300 of 411 posts

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#291

Post by akoboldfrying »

zahlman wrote: post #210
coldtea wrote: post #179

Earlier quoted context omitted.

Not if an LLM over chat can fool most people they're talking to a human (which it can), where the TI-99 speech synthesizer voice absolutely can not.

> Not if an LLM over chat can fool most people they're talking to a human (which it can) I keep hearing this claim, and yet I keep seeing LLM output which is trivially distinguished from human writing. I really can't understand how this gap persists; but then, there seem to have been at least some people who couldn't sniff out ELIZA, back in the day, too.

https://arxiv.org/abs/2503.23674

From the abstract: "When prompted to adopt a humanlike persona, GPT-4.5 was judged to be the human 73% of the time: significantly more often than interrogators selected the real human participant. LLaMa-3.1, with the same prompt, was judged to be the human 56% of the time"

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#292

Post by hgoel »

DanielHB wrote: post #226
eru wrote: post #144

Earlier quoted context omitted.

Yes, and we are still pretty early: AI is still advancing at breakneck speeds, and hardware is too.

Is the hardware really getting better? It feels performance per watt is not getting better at all which is the metric that will matter eventually when supply-demand stabilizes. As it is, it seems the improvements are about making the hardware cheaper (as in capex, not opex). This is just feels from me from what I hear on the news and see on the products though.

The DGX Spark apparently consumes up to ~150W while being able to run many models at decent speeds.

I think really good efficiency is possible right now, but the GPU makers don't want to make their consumer GPUs too good for AI - if the cards were more efficient, it'd be much easier to run multiple - while data center ones have a bunch of additional power overheads.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#293

Post by snapcaster »

Tade0 wrote: post #284
coldtea wrote: post #179

Earlier quoted context omitted.

Not if an LLM over chat can fool most people they're talking to a human (which it can), where the TI-99 speech synthesizer voice absolutely can not.

Does that mean it's intelligent? To me it just means they can brilliantly fake human conversation - the original design goal of Large Language Models. It's really easy to tell if you're talking to an LLM if you ask a question that requires actually knowing things, not going for the first search result of a tool call or whatever most popular answer was embedded in the weights. For this reason even the most sophisticat…

Why does this matter?

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#294

Post by cyanydeez »

fweimer wrote: post #135
refactor_master wrote: post #33

Earlier quoted context omitted.

Unlike cloud infra in general which offers things like automatic backups, regional redundancy, and effectively unlimited scalability, it seems like the value proposition of cloud LLM gets ever shakier. * Many businesses don't need frontier level intelligence anyway. * It's completely stateless. If your local LLM machine catches fire? Nothing was lost. Buy another.

Centralized inference can easily increase batch size, leading to huge efficiency gains in the usual scenario where most users have just one or very few session. Using local resources efficiently requires some way to increase the batch size. I'm not sure if we are there yet.

Centralization without proper controls against monopolization becomes sloth and gluttony. If american labs were constrained like china, their models would benefit.

The abstract benefits are quickly outstripped. The same way adding more highway lanes never improves gridlock. Its inducement.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#296

Post by lxgr »

vanviegen wrote: post #168
suprjami wrote: post #63

Earlier quoted context omitted.

So-called "caveman" thinking attempts to address this. The important part of "actually wait, I really need to XYZ" is just "XYZ". The model can attend to just "do XYZ" and produce almost the same vector modifications as full verbose "reasoning".

I don't think that's true. If a context contains a statement followed by something opposing that statement, that will confuse the model. So "actually wait, I really need to" is there to signal that the previous thinking may be flawed and that what follows is a new attempt. It's good to remember that LLMs have no more state then what they can derive from the context up til any point. So if that context is hard to inte…

As long as the model itself understands that when it says "foo bar" in its own reasoning trace it means "pursue theory foo; no, this didn't lead anywhere, let's backtrack and pursue theory bar", where's the loss in clarity?

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#297

Post by graceful6800 »

harhargange wrote: post #253

I had been planning to buy a GPU for Blender to compliment my 5950x CPU. The moment I ran the Qwen3.6-27b on my CPU, I arrived at the conclusion about the GPU I want. I saved some money and ordered the 7900xt-20gb for around 600 USD (instead of 7900xtx for 900USD, Nvidia out of question due to prices). I just ran the Qwen3.8-27b and asked it to benchmark itself. Here's the output: ====================================…

I run the same card and you're going to wish you'd gotten just a bit more VRAM :(

20GB is just barely not enough unless you want to live with 32k context, which just isn't enough.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#298

Post by sgt »

XCSme wrote: post #252
sgt wrote: post #248

Earlier quoted context omitted.

How about a complete rewrite that is not vibe coded? I'm only half serious. You're likely to use an LLM to help develop it, but don't let it take you for a ride. LLM's really struggle with data dense and robust designs. Have a look at https://diskprices.com/ - this is great UX because it's extremely functional, yet it has nearly no design to speak of. On a scale of data dense designs, this one would be on the extreme…

I am still pretty happy with some UX decisions I took: - the spotlight search, you can do ctrl+k and search "gpt vs claude vs google" and it automatically shows the most relevant comparison - drag to reorder compared models - filter leaderboard by multiple comma separated needles, so you can see for example all qwen and kimi models only - the showcases page is quite cool A lot of invisible UI/UX features are there, w…

Fair enough. I'll check it out in the next few weeks to see how the site evolves.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#299

Post by sourcecodeplz »

shifto wrote: post #290

Local Qwen3.8 27B pointed to my local Silverbullet wiki coupled with a harness makes it so smart in my environment. It knows my network and homelab better than me. It can find things in an instant, troubleshoot problems on its own. Magic.

what do you run and what in your homelab? if you dont mind responding

i just run a spare pc for browsing and screenshots with proxies.

Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things

#300

Post by hbogert »

shifto wrote: post #290

Local Qwen3.8 27B pointed to my local Silverbullet wiki coupled with a harness makes it so smart in my environment. It knows my network and homelab better than me. It can find things in an instant, troubleshoot problems on its own. Magic.

Yeah even qwen 3.5 9B on my VERY old RX 580 from 2017 can make sense of my homelab given a few crucial pointers in a agents.md. I think that's wild.
Post reply on HN