Qwen 3.6 is ~$2/m tok, 3.8 should be drop in replacement. Gemma 31B is $0.34/m tok. The price differential on these models is massive on openrouter.
Why are you comparing a 2.4 trillion Max model to a 31 billion model?
Qwen 3.8 27B
271–280 of 848 posts
Re: Qwen 3.8 27B
#272Earlier quoted context omitted.
Unsloth publishes KL divergence numbers which measures how much the quantised probability distribution changes vs unquantised: https://unsloth.ai/docs/models/qwen3.8#quantization-analysis It's a bit bare at the moment, I assume they are going to add further detail later (eg comparison to other quants), similar to their other releases.
KL divergence is nothing close to a replacement for benchmarks. As flawed as benchmarks are, KL divergence is a barely useful signal. The fact that Unsloth only just started publishing KL divergences shows how unserious the quantization space is.
You're very right about KL divergence. I spent a couple days playing with the Gemma 4 models. That's 10 separate models (varying weights, MoE, QAT or not, etc...) with identical tokenizers. I treated 31B at BF16 as the gold standard, feeding Wikipedia snippets, and anthropomorphizing a bit:
Gemma 4 31B: "Um, if I really said all of that, I guess I'd say this next"
Gemma 4 26B: "Dude, I would've said completely different stuff" (large divergence)
Gemma 4 12B: "Umm, there's zero chance I would've said some of this" (INFINITE divergence)
Gemma 4 E4B and E2B: "Derp derp, I'm happy to say almost anything" (lowest divergence)
For models which are chat trained, they simply would not recite Wikipedia, so the divergence is almost meaningless. I thought about capturing a realistic coding session and trying to use that as the corpus, but you need to preserve the turn-based tokens and such, so I moved on to other things.
Re: Qwen 3.8 27B
#273The $1500 Intel B70 with 32GB of VRAM can run this model at max context with good performance, btw. If you don't want to drop $5-10k for running DeepSeek this is your best budget option for local refactor/small scale dev help
I understand the B70 is a bargain vs AMD and especially nVidia offerings, but to me it feels like I would be buying something that would feel too limited in less than a year. 48G would be much more confortable. And I know the 96G nVidia cards are selling for over 10k$. The future can’t arrive fast enough!
32GB is perfect for models around 30B parameters. Since qwen has really hit the spot with their 27B dense models, I think it's a good bet. Also, 32GB is enough for other tasks such as image/video generation and loading multiple smaller specialized models
Re: Qwen 3.8 27B
#274Earlier quoted context omitted.
How much does it score though? 0% would be 4% less if Opus was at 4%. Unless you mean relative fraction not percentage points - but people usually mean percentage points in such situations.
0% is not 4% less than 4%, that would be 3.84%. 0% is 4 percentage points (pp) less than 4%.
[dead]
Re: Qwen 3.8 27B
#275Earlier quoted context omitted.
Unfortunately, that chip just doesn't really have the memory bandwidth to run this (or nearly any) model at acceptable speeds. I have the exact same chip (M4 Max 128GB) and I've been trying to optimize a completely purpose-built implementation with Fable and this is just not possible. Even if you could reach the full 576GB/s, it's just physically impossible to exceed these numbers with the model's architecture: 2 bpw…
What is bpw? Also whats your cutoff for 'acceptable' speed? I would have said 25tok/s.
bits per weight
Re: Qwen 3.8 27B
#276Earlier quoted context omitted.
So if I hire an artist and am a motivated individual, have an idea for a painting, have background knowledge about paintings and have taste in paintings and can offer a critique of the painting as the artist paints it, then somehow I created the painting? Absurd logic. The AI built the website.
Nothing absurd about that. What do you think an "Executive producer" is? A "Director" ? Does Peter Jackson get credit for creating the Lord of the Rings Trilogy films? Christopher Nolan for his films? But did he make them ? No, it was the collective effort of thousands of individuals all working under their direction. Just like if somebody creates software today, and the end result is generated by the collective effo…
The director receives accreditation for directing the film, not creating it.
Re: Qwen 3.8 27B
#277Earlier quoted context omitted.
just to clarify. yes YOU built it. just because you used some tool doesn't mean the idea, prompting, reprompting, babysitting was not your creative input and effort. put differently, if you put a random person infront of whatever model you used (say, a 50yo receptionist at a pharmacy in india), they would not have been able to create that, because they would have lacked the motivation, idea, background knowledge, tas…
Some people will now argue it was the chisel—not Michelangelo—who created David.
The idea guys will really think themselves gods, now.
Re: Qwen 3.8 27B
#278I hope really badly that we'll get a new 35B A3B or similar MoE model! I also miss the Qwen 3 Coder Next, which was 80B A3B, there are quite a few use cases where a non-dense model Also alternate link for viewing the images without signing in: https://xcancel.com/Alibaba_Qwen/status/2088280182356611304
I'm still confused about Qwen 3.6 35B A3B. Everything I read said that the 27B model performs better at coding tasks, so what's the purpose of the 35B model?
Re: Qwen 3.8 27B
#279Earlier quoted context omitted.
I wish each quant was benchmarked on the same tests as the original network so we could compare their performance
Unsloth publishes KL divergence numbers which measures how much the quantised probability distribution changes vs unquantised: https://unsloth.ai/docs/models/qwen3.8#quantization-analysis It's a bit bare at the moment, I assume they are going to add further detail later (eg comparison to other quants), similar to their other releases.
The talk around KL divergence is oversold. People talk about it as if it’s not a benchmark, but at its core it is in the same neighborhood! You get a different KL divergence number on different “calibration datasets”, so in other words it’s data-dependent. It is NOT a universal guarantee about the fundamental divergence of a model.
Re: Qwen 3.8 27B
#280Earlier quoted context omitted.
So if I hire an artist and am a motivated individual, have an idea for a painting, have background knowledge about paintings and have taste in paintings and can offer a critique of the painting as the artist paints it, then somehow I created the painting? Absurd logic. The AI built the website.
You know how many pieces of art Damien Hurst creates himself Vs his studio assistants creating them under his direction? For example, of his 1500 spot paintings, he only actually made 5 of them. It's not uncommon at all for artists to work this way.
He still didn't create the paintings, he offered instructions / feedback to the artists that created them. Whether or not he is credited with authoring them is an entirely separate issue.