Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

751–760 of 848 posts

Re: Qwen 3.8 27B

#751

Post by marius_ »

spijdar wrote: post #464

I wonder how this practically compares with Muse Glimmer, especially quantized. I've got an RX 7900 XT (20GB of VRAM) and I can run glimmer with a full 128k context window with the draft model at 65-80 tok/s. This model, on the other hand, I get about 30 tok/s with a 30k context. Raising the context or loading the draft layers for MTP drops performance to 9-15 tok/s. So I wonder how big the "real world" delta between…

From my limited tests, Qwen has better reasoning which is a bummer because Muse Glimmer is literally twice as fast.

Re: Qwen 3.8 27B

#753

Post by frumiousirc »

simonw wrote: post #419

Absolutely the best pelican I've seen from a model that runs on my laptop: https://tools.simonwillison.net/markdown-svg-renderer#url=ht... Bicycle is the right shape. Pelican beak is excellent. Nice background. Most importantly, the pelican has one leg on each side of the bicycle - that's very rare. (No chain on this bicycle though - in the reasoning trace it says "already chainstay... skip chain detail; maybe a smal…

> Bicycle is the right shape. Pelican beak is excellent. Nice background.

Relative to other results I agree. But on an absolute measure, there is not a single element in the current bicycle that is real-world accurate and many elements are omitted or non-physical (eg, the transparent seat tube top, entire lack of a head tube).

Consider a series of followup benchmarks.

With a fresh context of the LLM under test, ask it to generate a list of findings for how the pelican-on-a-bicycle SVG that was produced is inaccurate compared what the real world scene might appear, accepting for the limitations of SVG as a medium. Then, feed back the list of findings to the original context for a second try. The benchmark can stop here by humans looking at the result and forming their own conclusion.

Next phase is to repeat the analysis phase using the 2nd context to determine what findings were satisfied and what new inaccuracies are found. These two differences can form a second benchmark.

Last phase is to iterate with the goal to drive the number of findings to zero.

Re: Qwen 3.8 27B

#754

Post by spider-mario »

Casteil wrote: post #82

One thing a lot of people don't seem to factor when hyping Qwen is how much models like this tend to 'overthink' with seemingly endless 'second guessing'. 3.8 seems no different from what I've tried thus far. As capable as it is, it's hard to justify using it when a competing model (e.g. Gemma4:26b-a3b) can consistently achieve the same or similar response with only 1/10th as many 'thinking' tokens, achieve much high…

> As capable as it is, it's hard to justify using it when a competing model (e.g. Gemma4:26b-a3b) can consistently achieve the same or similar response with only 1/10th as many 'thinking' tokens, achieve much higher tokens/second, and take a small fraction of the time.

Not my experience at all. On my M1 Max MacBook Pro, Qwen 3.6 35B-A3B provides better responses in similar time (~60 tokens/second, vs. 50 for Gemma 4 26B-A3B). 3.8 27B is slower, being a dense model, but so are dense Gemmas.

Re: Qwen 3.8 27B

#755

Post by greenicon »

gaigalas wrote: post #413
zazibar wrote: post #410

Earlier quoted context omitted.

Good news, MTP support is already included in this release. Not sure why they haven't made this clearer.

I don't see an MTP entry on Unsloth though. Maybe it's not available in a lower quant I need for my poor GPU.

It's included in the model gguf itself.

Re: Qwen 3.8 27B

#756

Post by c16 »

Big thank you to the Qwen team. 3.6 A3B was shocking good, and now I'm hoping they release an 3.8 A3B model too.

Edit: Having used qwen3.8:27b-mlx on MBP M4 64GB, I get around ~45 tok/s. A3B would be great for smaller devices, but it's definitely usable. As I understand it it's a mixture of MLX and MTP.

Re: Qwen 3.8 27B

#757

Post by lostmsu »

nullc wrote: post #674
lostmsu wrote: post #154

Earlier quoted context omitted.

> The fact that Unsloth only just started publishing KL divergences shows how unserious the quantization space is. Just wanted to say that this is a very important point that I totally agree with. People are obsessed with KL divergence, but it is yet to be demonstrated to be a descent proxy for agentic coding benchmarks.

It's a cheap to evaluate proxy for totally broken or not, which is a good start. It also has a lot of resolution and not a lot of noise. Better would be multi-turn benchmarks with tools but getting good precision and accuracy for that is hard and computationally expensive.

> proxy for totally broken or not

You can't really know that either.

Re: Qwen 3.8 27B

#758

Post by aka-rider »

DanielHB wrote: post #51
onlyrealcuzzo wrote: post #21

If the benchmarks don't lie, this is getting very close to Opus 4.6 capability - which was the turning point for me for when AI was "good enough" that it became very hard to justify not using it. I'm sure there's some benchmaxxing going on, and some things you get only with a a larger model. But I'm feeling pretty confident if not by Gemma 5 than by mid 2028 we'll have local models that are almost always as good as O…

What kind of things you only get with a larger model?

Smaller models are overconfident and have a hard time to self-correct.

If it’s stuck, usually that’s it.

Bigger models “understand” better, both the prompt and the contents. If you will try to read a paper together with a smaller model, the difference is immediately obvious.

Bigger models will “forget” and drift much less.

Re: Qwen 3.8 27B

#759

Post by dofm »

So far what I am seeing in my seemingly simple "Wordpress last-login plugin" test is that in xhigh reasoning mode (the default, seemingly) it overthinks so badly that it writes terrible bushy code with edge cases caused by going down rabbit holes.

In "medium" reasoning mode, you get the classic Qwen wait/actually thinking loops you see in 3.6 that I guess will need to be interrupted in the way others do already with an over-thinking guard proxy.

(In one of my test runs it is now on "OK TRULY FINAL APPROACH" after having got through "FINAL FINAL APPROACH". Can relate)

It gets stuck in a thinking loop regarding the WordPress API and (resolvable) ambiguity in my prompt, that I guess might be resolvable with a custom skill with hints on how to look it up (and maybe with the devdocs MCP).

In Low reasoning effort mode it flies through the task and writes pretty solid code. So maybe it is me overthinking what is needed here...

Re: Qwen 3.8 27B

#760

Post by pram »

dofm wrote: post #749
pram wrote: post #615

Earlier quoted context omitted.

Glimmer works really well as an "explore" agent model (like in Opencode.) It seems to be extremely efficient at searching and collating that info, and executing commands. From my testing so far, Qwen 3.8 is better at code but it tends to meander and take forever if it has to look in a lot of places. Glimmer will use like ~1k tokens to formulate a plan and Qwen 3.8 will routinely go over 10k

Have you tried turning down the new Qwen's reasoning effort level from xhigh, which it defaults at? LM Studio isn't exposing a dropdown for this, at least with the unsloth build. Unsloth Studio / Desktop does.

These templates actually fix the effort selection for LM Studio/3.8

https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

Post reply on HN