Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

661–670 of 848 posts

Re: Qwen 3.8 27B

#661

Post by nullc »

dofm wrote: post #550
CMay wrote: post #518

Credit where it's due. Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. It took 5x as many tokens to do it and 12m30s with MTP enabled, but it did do it. Gemma 4 reasoned through it more implicitly, while Qwen 3.8 reasoned more explicitly. Laguna and Muse Glimmer failed hard on it, though they're useful for other tasks. The VRAM usage see…

I was quite impressed by Muse Glimmer, and while I am sure people will observe that it is less good on benchmarks, my first experiences with this new 27B have been somewhat exasperating, whereas testing Muse Glimmer was rather fun. I have not tested either in an agentic context, mind you.

I wouldn't find glimmer interesting except that it has much less memory usage per token of KV than Qwen. So I can get 24x concurrent glimmer on 2xRTXA6000 (with 128k context) where I can only get 6 Qwen 27b. This means I can get something like 4x the aggregate tokens/s out of glimmer.

For some usages that speedup more than makes up for it being inferior to Qwen intelligence wise.

Re: Qwen 3.8 27B

#662

Post by nullc »

Der_Einzige wrote: post #180
skrebbel wrote: post #178

Earlier quoted context omitted.

I'd love to understand this more. Are you saying the Qwen team spends their very impressive human and compute resources on publishing these amazing models and then botches the chat template with mundane bugs? Like maybe I just misunderstand what's the hard part but wouldn't you assume that people who can put together an impressive model can also write a proper jinja chat template for it?

Yes yes, oh god yes. They also spread FUD in the form of terrible recommended sampler settings. If you're using llamacpp, turn on top-n-sigma with sigma of 1, turn off top-p/top-k. You'll thank me later.

Diverging from the sampler used in RL training is not good for long multi-turn results-- it's a great way to knock models into reasoning loops that wouldn't otherwise.

Re: Qwen 3.8 27B

#663

Post by nullc »

anon373839 wrote: post #650
redox99 wrote: post #164

Earlier quoted context omitted.

Asking it factual information[1]. You just can't compress the entire human knowledge into a 30GB file. [1] Without searching the internet. And even if you allow it, you'll get much worse results because search means browsing and parsing the top results, and search results are horrible, whereas internal knowledge from training encompasses the entire internet plus all books including very niche stuff.

> You just can't compress the entire human knowledge into a 30GB file Fortunately, that isn’t necessary! What LLMs need is a level of fluency with key concepts so that they can (1) make effective use of retrieval tools and (2) understand the material in the context window. 30GB-sized models can absolutely store enough knowledge to do this. Here is an example from the field of law. Most lawyers who have litigated cont…

> (1) make effective use of retrieval tools and (

A downside is that you can't just download a lot of that knowledge, vs with the weights the copyright infringement has been outsourced to the lab. Nor can you just search for the info because the internet as a whole is increasingly aggressive at blocking anything that looks like an AI agent.

I'd love to see more retrieval powered local AI-- I think it's an area that open source development could excel. ... but there are advantages of having the knowledge in the weights!

Perhaps what needs happen is for someone to make an "ultrapedia", an AI restatement of a huge library of reference works-- created expressly for the purpose of being a locally stored corpus for AI agents.

Re: Qwen 3.8 27B

#664

Post by pdude444 »

Do we really feel like it’s the governments job to regulate OPEN source AI.

At rely health, we use OSS models in a HIPAA complaint and SOC 2 complaint environment to take advantage of asymptotically $0 intelligence to provide best in class care navigation . This should be industry standard -

Re: Qwen 3.8 27B

#665

Post by nullc »

SwellJoe wrote: post #377
KronisLV wrote: post #10

I hope really badly that we'll get a new 35B A3B or similar MoE model! I also miss the Qwen 3 Coder Next, which was 80B A3B, there are quite a few use cases where a non-dense model Also alternate link for viewing the images without signing in: https://xcancel.com/Alibaba_Qwen/status/2088280182356611304

AgentWorld is a pretty recent MoE release from the Qwen folks with quite a bit better performance than 3.6, released around the same time as 3.7 Plus/Max, I guess. Not sure why it didn't get more attention, as it is definitely better than 3.6 35B A3B on all dimensions, but especially for agentic use. Still nowhere near the dense models (even 3.6 27B), but clearly an upgrade in the small Qwen MoE line. https://hugging…

HUH? AgentWorld is a simulation of the world (e.g. tools and programs) for use by an agent!

Re: Qwen 3.8 27B

#666

Post by nullc »

colordrops wrote: post #278
KronisLV wrote: post #10

I hope really badly that we'll get a new 35B A3B or similar MoE model! I also miss the Qwen 3 Coder Next, which was 80B A3B, there are quite a few use cases where a non-dense model Also alternate link for viewing the images without signing in: https://xcancel.com/Alibaba_Qwen/status/2088280182356611304

I'm still confused about Qwen 3.6 35B A3B. Everything I read said that the 27B model performs better at coding tasks, so what's the purpose of the 35B model?

35Ba3b is usable in plain cpu inference on a fast server, the dense model is MUCH slower. On GPU the 35ba3b is still around 2x the tok/s single threaded, which can be a good tradeoff for some applications.

Re: Qwen 3.8 27B

#667

Post by nullc »

xlayn wrote: post #466

The unsloth Q8kxl https://huggingface.co/unsloth/Qwen3.8-27B-GGUF for some reason is looping and going crazy on the think part (I tried to search for an email to let the guys know but didn't find one)... I used the bartowski one and that one doesn't have that issue https://huggingface.co/bartowski/Qwen3.8-27B-GGUF?show_file_... that's using llama.cpp llama-server \ -m ~/somePath/Qwen3.8-27B-UD-Q8_K_XL.gguf \ -np 1 --…

I'm not having any looping.

> --temp 0.2

Looping is a common symptom of changing the sampler settings from what it was RL trained with.

Re: Qwen 3.8 27B

#668

Post by bitexploder »

CMay wrote: post #518

Credit where it's due. Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. It took 5x as many tokens to do it and 12m30s with MTP enabled, but it did do it. Gemma 4 reasoned through it more implicitly, while Qwen 3.8 reasoned more explicitly. Laguna and Muse Glimmer failed hard on it, though they're useful for other tasks. The VRAM usage see…

Have you tried Muse 30B yet? I have been impressed with it. I have Qwen 3.8 27B hammering away right now against Muse. And Muse is doing a little bit better.

Re: Qwen 3.8 27B

#669

Post by suprjami »

MrDrMcCoy wrote: post #610
suprjami wrote: post #561

Earlier quoted context omitted.

Temperature, top-up, top-k, min-p all control which token the model predicts next and how likely it is to select one token over the other. You might understand this as "The capital of France is..." and the model isn't always going to select "Paris". Sometimes it will start a descriptive sentence or even get the answer wrong. That selection of the next token is what these settings control, and lots of sub-optimal sele…

I broadly knew that about temperature, but lack the background in machine learning/statistics to differentiate top-n-sigma from top-k/top-p.

Top-K: example setting 20. Select only from the 20 most likely tokens.

Top-P: example setting 0.9. Select tokens whose probably accumulates to this number. So say you have tokens with 0.7 then 0.2 then 0.1, the last will not be selected because the first two tokens already accumulated to >=0.9.

Min-P: example setting 0.05. Don't select tokens less probable than this value. So a token with 0.1 would be considered, a token with 0.01 would not.

The purpose of all of these is to exclude very unlikely next tokens.

Re: Qwen 3.8 27B

#670

Post by bitexploder »

CMay wrote: post #635
dofm wrote: post #550

Earlier quoted context omitted.

I was quite impressed by Muse Glimmer, and while I am sure people will observe that it is less good on benchmarks, my first experiences with this new 27B have been somewhat exasperating, whereas testing Muse Glimmer was rather fun. I have not tested either in an agentic context, mind you.

Yeah, Glimmer is excellent. You don't really test Glimmer with one-shots, because it's explicitly designed for multi-turn solution finding. The way I see it, if I've got a task that could be done either agentic or requires a lot of context (for example, dumping 600KB of API documentation and another 300KB of codebase for a project) then I would reach for Glimmer easy and it seems like it could get there most of the t…

That is exactly how this model has worked for me so far. Muse on a one-shot task will get to 80%. And if you even nudge it and say, "Hey, finish up," or "Review the syntax," boom, it's done. And I'm getting 20 t/s with Ollama on a MacBook M5 Pro with 48GB of RAM. It is a seriously impressive little model.
Post reply on HN