[flagged]
Qwen 3.8 27B is excellent, but it defaults to overthinking things
241–250 of 411 posts
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#242Earlier quoted context omitted.
Is that similar to what ggerganov is talking about here? https://x.com/ggerganov/status/2089214161884414147
Yes - it would seem so :) I did make a PR to the official llama-cpp repo some time back (about a month or so), but abandoned it as there seemed to be too much community concern that the mechanism would degrade model performance... Perhaps i'll polish it up and put some effort into benchmarking and revive the project in the near future.
would be great as an opt-in though
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#243Earlier quoted context omitted.
Yes, and we are still pretty early: AI is still advancing at breakneck speeds, and hardware is too.
Is the hardware really getting better? It feels performance per watt is not getting better at all which is the metric that will matter eventually when supply-demand stabilizes. As it is, it seems the improvements are about making the hardware cheaper (as in capex, not opex). This is just feels from me from what I hear on the news and see on the products though.
Solar power and batteries are getting cheaper and cheaper at the moment. So Watts should become cheaper in the long run.
Especially when chips are becoming cheaper (in the capex sense), then you can afford to only run them when power is cheap.
Btw, from where do you take the notion that performance per Watt ain't increasing? We are also still using what's more or less general purpose GPU hardware; we could get a lot further if we were willing to specialise more. Which would be the natural avenue to explore, if progress in general purpose hardware slows down. Google is already looking.
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#244Earlier quoted context omitted.
The model itself is excellent, the defaults are bad. As Simon and other pointed out, medium is great. Reminds me of Gemma4 and the official (or at least popularly used around launch) Jinja templates being wrong and broken for tool calling.
Seems less like "bad"/broken defaults and more defaults tuned to the max for benchmarks. All the positive PR from "Opus 4.6 level" online buzz is well worth the minor annoyance from taking a half hour to solve a simple problem since a user just needs to turn down the reasoning knob if it bothers them.
It’s both: The default is bad, but not by accident. Since they certainly chose this default intentionally to be evaluated by it, they entirely brought it upon themselves for it to be evaluated as slow, overthinking and overcomplicating things.
It’s a similar level of dishonesty as trying to conflate “starts at” vs. “tested configuration” car prices.
We change the incentive to do this by evaluating it exactly as advertised.
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#245I wonder if this can be fixed with LORAs.
This morning I tried experimenting with this ThinkingCap lora I found someone made for 3.6 https://huggingface.co/signsur4739379373/Qwen3.6-27B-Thinkin... ThinkingCap is a 3.6 27b finetune that claims to halve thinking tokens while maintaining the same output quality. I've used the model a lot and I'd say it holds up. Since 3.6 has the same architecture as 3.8, the lora can be applied. With the prompt "create a fancy…
"Generate an SVG of a pelican riding a bicycle" tests:
https://gistpreview.github.io/?815466e3208746488d47679949b68... - 33170 tokens
https://gistpreview.github.io/?815466e3208746488d47679949b68... - 18125 tokens
https://gistpreview.github.io/?815466e3208746488d47679949b68... - 12960 tokens
Scale 35 felt a bit noisy and incoherent, but 30 seemed nice. (they use the same seed, but idk how reliable seed in llamacpp is)
I use a python test script that captures the answer and renders it to a html page along with the llama-cli log, launch parameters, the chat log, and the python script itself for maximum transparency. :)
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#246It's said that some evaluations show that the drawing effect is not good after engaging in high-intensity thinking
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#247Earlier quoted context omitted.
At the bottom of this very submission are a bunch of dead comments that are very obviously LLM-generated.
Ok, you and I can easily spot LLM text. So what? The Turing test has still been passed, as is clear by people falling in love with ChatGPT, not believing something is AI, and by continuously claiming this or that is a bot. People, many of them at least, cannot make this distinction anymore. You can, I can, but people as a whole are having problems with that.
>You can, I can
Even this (assuming it's even true) will likely not be true in some near-term future.
>continuously claiming this or that is a bot
I see it as a contemporary form of religious thinking. Like (say) pilgrims seeing blood on a statue of the virgin, plenty of people are now seeing the hand of AI in everything they read. If you want to see something hard enough, it tends to become magically visible.
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#248Earlier quoted context omitted.
Btw, unrelated, but this is the kind of Vibe Coded AI slop design I see a lot these days. Every single thing on this page has a different color, formatting, and it's just painful to look at.
Thanks for the feedback! That might be my fault, I am not a designer, and I asked for most UI decisions. I tried to use colors to diferentiate models, the site is very data dense and it's hard to make everything readable. I've spent hundreds of hours building it, not sure if I would call it slop, but I just suck at design, lol Any suggestions on how to improve it?
How about a complete rewrite that is not vibe coded? I'm only half serious. You're likely to use an LLM to help develop it, but don't let it take you for a ride. LLM's really struggle with data dense and robust designs.
Have a look at https://diskprices.com/ - this is great UX because it's extremely functional, yet it has nearly no design to speak of. On a scale of data dense designs, this one would be on the extreme side, so I guess if you ask the LLM to meet you somewhere in the middle.
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#249“The fact that a 17GB file can do all of this stuff on my home machines is a miracle. Once again, I’m delighted and amazed at how much progress local models have made this year.” I think that should be the blinking headline - this shows what can be done with consumer hardware.
For me that moment was Gemma 4 12B QAT. You're not suddenly going to start throwing your hardest programming problems at Gemma 4 12B QAT, it is still 15B parameters less. It's more that, aside from pelican art which isn't what local models are for, I didn't see anything on Simon's post that it couldn't assist with or largely succeed at. It can run 80-100t/s on a laptop, can understand images natively and do bounding…
> transcribe or translate anything you say
Is it multimodal? How do you do transcription with it?
Re: Qwen 3.8 27B is excellent, but it defaults to overthinking things
#250Earlier quoted context omitted.
Thanks for the feedback! That might be my fault, I am not a designer, and I asked for most UI decisions. I tried to use colors to diferentiate models, the site is very data dense and it's hard to make everything readable. I've spent hundreds of hours building it, not sure if I would call it slop, but I just suck at design, lol Any suggestions on how to improve it?
How about a complete rewrite that is not vibe coded? I'm only half serious. You're likely to use an LLM to help develop it, but don't let it take you for a ride. LLM's really struggle with data dense and robust designs. Have a look at https://diskprices.com/ - this is great UX because it's extremely functional, yet it has nearly no design to speak of. On a scale of data dense designs, this one would be on the extreme…
I do try to guide it, I don't think strictly telling LLMs what to do works best. I like letting the LLM do what it thinks it's best (supposedly it is a lot smarter than me), and then guide it in the directions I think it should go and fix the flaws I notice.
I checked out that link on mobile, but it's unfortunately unusable. I will check it out later on desktop, thank you.