Surely this is great for an end user: the taste as to “when” and to what degree a model should “think” is now entirely in the fine tuners hands
Qwen 3.8 27B is excellent, but it defaults to overthinking things
111–120 of 411 posts
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#112I find that a lot of the recent allegedly great open models are cranking their reasoning way further than I find reasonable for interactive use. I’m writing this while waiting for the new Deepseek V4 Flash to finish its task, which is taking way longer than the older version. What gets reported is always the benchmark result, but rarely the real-world trade-off made to achieve it. That’s an obvious incentive for the…
The model itself is excellent, the defaults are bad. As Simon and other pointed out, medium is great.
Reminds me of Gemma4 and the official (or at least popularly used around launch) Jinja templates being wrong and broken for tool calling.
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#113I feel like the current “reasoning” that LLMs are doing has got to be a dead end eventually. Every time I have to read another answer with “but wait” and “Actually,” as they “reason” their way to a (sometimes) better answer, I feel like there’s got to be a way to just shortcut to the actual correct answer instead of burning all these token going in circles mimicking actual thought
Chain-of-thought output shouldn't be taken literally. The tokens are a substrate for computation, not necessarily evidence that the model is wasting time and electricity by gratuitously second-guessing itself over and over.
You can see evidence of this phenomenon in models dating back to the OG Deepseek R1. It was common to see the model talk itself out of the correct solution in the block, or fail to reach it at all, only to produce a correct answer in the response. And vice versa; it was also common to see it reason its way to the right answer and then fail to follow through in the response.
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#114To me, the amazing thing is that we now have local models that rival the reasoning of high end models from about a year ago. I hope this trend continues.
Unlike cloud infra in general which offers things like automatic backups, regional redundancy, and effectively unlimited scalability, it seems like the value proposition of cloud LLM gets ever shakier. * Many businesses don't need frontier level intelligence anyway. * It's completely stateless. If your local LLM machine catches fire? Nothing was lost. Buy another.
I mean, I might be missing something, but isn't part of the idea with cloud infra that you can scale down as well?
Large orgs with significant demand might go out and buy local LLM hardware, but most businesses probably don't want to bother dropping $2k on a box with a beefy GPU and would rather just pay the lowest subscription tier so their employees can occasionally make queries.
Plus, you know, the whole economies of scale thing. Local LLM has a lot of privacy and independence benefits, but I'm not really seeing the world where it becomes more energy- or cost-efficient to buy your own hardware (and use it 1% of the time) versus sharing a giant machine, or even the same machine, in a datacenter (where it has a much higher utilization factor).
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#115“The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.” I think that should be the blinking headline - this shows what can be done with consumer hardware.
Like the old proverb: "The marvel is not that the bear dances well, but that the bear dances at all."
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#116“The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.” I think that should be the blinking headline - this shows what can be done with consumer hardware.
Full agree. I until very recently thought AI tools of today were limited to prohibitively expensive high end hardware hosted in data centers.
I was surprised and amazed to get "decent" (with the expectations set right / low) coding performance out of Qwen3.5-9B on a decidedly medium end Radeon 9070 paired with a 5700x3d and 32GB of DDR4 RAM.
We can finally reason with and "talk" to our hardware.
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#117This evolution of models that doesn't only favours big US corps is just good for humanity
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#118“The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.” I think that should be the blinking headline - this shows what can be done with consumer hardware.
I thought the same thing, and I generally do a lot of animation in my work, and the results in motion graphics with Qwen are impressive, I really fell in love with it
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#119I find that a lot of the recent allegedly great open models are cranking their reasoning way further than I find reasonable for interactive use. I’m writing this while waiting for the new Deepseek V4 Flash to finish its task, which is taking way longer than the older version. What gets reported is always the benchmark result, but rarely the real-world trade-off made to achieve it. That’s an obvious incentive for the…
[flagged]
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#120It won't satisfy the people who just want to drop a model into their existing toolset and run, but I think there are a lot of ways to deal with this overthinking problem. For instance, it's a step backward, but I put {"reasoning_effort":"none"} and led it by the nose: User: We're going to make . Please create a plan, but do not write code yet. Agent: User: Now please follow that plan and write the code. No other chat…
I'm doing much the same, avoid the long thinking loops and instead have more iterations on the plan with reviews from different angles.