Is there any way to turn off thinking if I'm using Ollama? In my particular case, the Ollama API (the software I want no-think for is tied to Ollama's bespoke API). If not, I'll stick to 3.6 for the time being...
Qwen 3.8 27B
551–560 of 848 posts
Re: Qwen 3.8 27B
#552Earlier quoted context omitted.
> I've settled on GLM-5.3 (formerly Deepseek v4 pro 0813) for architecting Dude, GLM-5.3 released _today_. The phrasing "I've settled on" is incorrect for this context.
I'm generally extremely skeptical about a lot of the model hype that show up in comments. Except when there is an extreme mismatch the performance, quirks and quality of these things are difficult to nail down. You wouldn't know that from the comment section of every single release . I think some of these are excited, eager users always ready to hype up the new thing. The same crowd that previously would constantly p…
I strongly suspect that many of these accounts you think might be bots from the labs are just people who only have one interest.
One thing I have noted a lot more of is that there is comparative fanboying going on. Like "this model has done badly, my favoured competition has a model out soon that will beat this in every way" — comparing a released product to unverifiable hopey claims about an unreleased product.
It's tempting to assume that is bot stuff, but if you've been around any other "hot" technical hobby online (cameras, phones, 3d printers, whatever) you will know it's not. It's just fans aligning into teams, some of them laconic and amusing, some of them overkeen and toxic.
Re: Qwen 3.8 27B
#553Credit where it's due. Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. It took 5x as many tokens to do it and 12m30s with MTP enabled, but it did do it. Gemma 4 reasoned through it more implicitly, while Qwen 3.8 reasoned more explicitly. Laguna and Muse Glimmer failed hard on it, though they're useful for other tasks. The VRAM usage see…
> The VRAM usage seems way less efficient than Gemma 4 or Glimmer though
Maybe it's implicit that you're using llama.cpp (although you don't mention GGUF), but it's hard to reach concrete conclusions about the model architecture based on one implementation in one runtime.
Re: Qwen 3.8 27B
#554Earlier quoted context omitted.
Luna is twice the price of Deepseek V4 Flash 0731, and less capable :/
How is it less capable? I get being cheaper, but I’m pretty sure Luna is the stronger model?
Benchmarks are basically tied
https://artificialanalysis.ai/models/comparisons/deepseek-v4...
Re: Qwen 3.8 27B
#555Earlier quoted context omitted.
Yes for the first question. Google of all companies didn't even get it right with Gemma for a while until recently. For some reason it doesn't seem like people can actually get these templates right.
Is the chat template used at all when they benchmark the model?
It’s unlikely they are benchmarking that downstream. They probably have private benchmark scripts with purpose-specific run telemetry and logging, etc.
I’m sure there is some basic testing but it might be agentic (LLM likes its own output) and maybe just some human smoke tests.
Re: Qwen 3.8 27B
#556As usual, the Jinja templates are messed up so use this [0] to reduce or turn off thinking, fix tool calling, keep a 100% KV cache hit rate, etc. [0] https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates
I'd love to understand this more. Are you saying the Qwen team spends their very impressive human and compute resources on publishing these amazing models and then botches the chat template with mundane bugs? Like maybe I just misunderstand what's the hard part but wouldn't you assume that people who can put together an impressive model can also write a proper jinja chat template for it?
You have understood correctly.
One really would think these companies (including Google) who spend many millions of dollars on compute could write a few hundred lines of Jinja correctly, so their investment works optimally or at all.
But they don't.
Then a couple of individuals on HuggingFace fix it, either a 2-person startup like Unsloth or a volunteer like froggeric.
I also don't understand how this repeatedly happens.
Re: Qwen 3.8 27B
#557Earlier quoted context omitted.
Reduce or turn off thinking: https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates
In my experience with Qwen3.6 35B-A3B, disabling thinking made the model generate inaccurate replies. Ask it for the recipe of egg salad and it gives you the recipe of an omelette. Did I miss something, is it possible to have that model be reliable without thinking?
The official docs recommend different sampler options for thinking off. Have you tried adjusting those?
Re: Qwen 3.8 27B
#558There's a real change (compared to 3.6) in the way it writes in thinking — it drops words like "to" and "we" in "We need to", talks generally in note form, drops the/and all over the place, avoids "for". "Need be helpful concise", "Need maybe not overdo", "Need ask!" Almost caveman. I have an (unsourced, vague) suspicion that this rather unique thinking trace pattern is actually hobbling the MTP predictions, which se…
This has been common in Kimi K3, and now DS Flash/Pro as well. Evidence of cross-training?
Re: Qwen 3.8 27B
#559Earlier quoted context omitted.
Didn't DeepSeek go up 2x?
The price on DeepInfra and most providers on OpenRouter has not changed... if anything, it's gotten cheaper. All of this DeepSeek price increase stuff is related to their specific API, not the entire market.
Looks like it's changed now, OpenRouter shows providers having the updated DS4 pricing.
Re: Qwen 3.8 27B
#560Earlier quoted context omitted.
Yep. These small models are actually worse than GPT 3.5 at some tasks (like recalling facts). You can definitely make models smarter at specific tasks (like tool calling, coding) but you can't compress the entire human knowledge into a 30GB file. It's just not enough bits.
That's a good sign. Ideal local model would not know stuff like who Britney Spears is, best to leave precious weights for something useful. Of course the line is very blurry but I'd be perfectly happy with local model that doesn't know anything about history, geopolitics, art or even biology etc. just coding, operating systems etc.
At one point there were specific -Coder release of Qwen e.g. 2.5 but they dropped that, still wondering how much better 3.6-Coder or 3.8-Coder would be when they ignore everything else