Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

671–680 of 848 posts

Re: Qwen 3.8 27B

#671

Post by fwipsy »

lwansbrough wrote: post #515
simonw wrote: post #421

Earlier quoted context omitted.

For anyone who followed yesterday's Gemini 3.7 Flash pelican which rendered in Safari but not in Firefox or Chrome... https://news.ycombinator.com/item?id=49289112#49290012 . - that turned out to be my fault, not the model. My SVG rendering software was stripping some attributes. Here's the Gemini 3.7 Flash pelicans in the fixed renderer: https://tools.simonwillison.net/markdown-svg-renderer.html#u...

Perhaps this isn’t a new observation but the problem with LLMs is very clear with these. It’s a nice microcosm. The LLM will draw a fish companion (unprompted!) with a nice gradient but won’t get the pelican’s feet right. It’s obviously a problem of fundamental understanding and demonstrates that reasoning is more “directionless rigour”.

I'd like to see a human create a better pelican SVG without being able to look at the results. LLMs are language models, not (natively) vision models*. That's why it's a good benchmark: it engages LLM's logical reasoning in a way that we can check visually. The fact that they make errors which can be spotted visually, doesn't prove that LLMs are so far behind humans on language/logic tasks.

Besides which, "fundamental understanding" isn't a binary. Humans can also have a deep understanding of a subject and still make errors. (I do, anyways.) I do not think "fundamental understanding" is itself well enough understood to say definitively that LLMs do or do not have it.

*Some models have multimodal capabilities but these are separate weights, I believe it's only engaged in processing images. From the reasoning trace, the model never rendered the result -- only instructed the user to do so. I welcome correction if I'm wrong, I only have a surface-level understanding here.

Re: Qwen 3.8 27B

#672

Post by suprjami »

fragmede wrote: post #624
MrDrMcCoy wrote: post #610

Earlier quoted context omitted.

I broadly knew that about temperature, but lack the background in machine learning/statistics to differentiate top-n-sigma from top-k/top-p.

So do I, but we live in the future: https://chatgpt.com/share/6a7fc3d2-39f4-83e8-a7c6-825ddfb5e7...

You might find this article relevant: https://news.ycombinator.com/item?id=49151933

Re: Qwen 3.8 27B

#673

Post by yogthos »

redox99 wrote: post #164
DanielHB wrote: post #51

Earlier quoted context omitted.

What kind of things you only get with a larger model?

Asking it factual information[1]. You just can't compress the entire human knowledge into a 30GB file. [1] Without searching the internet. And even if you allow it, you'll get much worse results because search means browsing and parsing the top results, and search results are horrible, whereas internal knowledge from training encompasses the entire internet plus all books including very niche stuff.

[dead]

Re: Qwen 3.8 27B

#674

Post by nullc »

lostmsu wrote: post #154
zargon wrote: post #114

Earlier quoted context omitted.

KL divergence is nothing close to a replacement for benchmarks. As flawed as benchmarks are, KL divergence is a barely useful signal. The fact that Unsloth only just started publishing KL divergences shows how unserious the quantization space is.

> The fact that Unsloth only just started publishing KL divergences shows how unserious the quantization space is. Just wanted to say that this is a very important point that I totally agree with. People are obsessed with KL divergence, but it is yet to be demonstrated to be a descent proxy for agentic coding benchmarks.

It's a cheap to evaluate proxy for totally broken or not, which is a good start.

It also has a lot of resolution and not a lot of noise. Better would be multi-turn benchmarks with tools but getting good precision and accuracy for that is hard and computationally expensive.

Re: Qwen 3.8 27B

#676

Post by ferrouswheel »

tonis2 wrote: post #325
ferrouswheel wrote: post #243

Earlier quoted context omitted.

But why would you use a model to store factual knowledge, that is stupid. We want intelligence, not a database.

Models cant make any decisions if they have 0 idea that the feature exist in this language or in some general fact. For example you would tell a model hey, become an expert in this language for me, search it online, it would still need to learn it and download the data to it's context and then increasing the memory usage, there's no way around it.

Implicit knowledge is different from remembering every church that exists in the state of NSW Australia

Re: Qwen 3.8 27B

#677

Post by [deleted] »

MrDrMcCoy wrote: post #625
bilekas wrote: post #618

Earlier quoted context omitted.

> I went to Huggingface, went to the Unsloth org, as they tend to be the best, went to the Model page, and went to the "Files and versions" tab. "I went to youtube, gmail, ycombinator, deliveroo, then I went to another site I randomly chose, because they're the best, duh" Okay. > Unsloth AI is a very popular, highly reputable organization that takes upstream model files And How am I supposed to know that arriving to…

> "I went to youtube, gmail, ycombinator, deliveroo, then I went to another site I randomly chose, because they're the best, duh" The path I described is all within HuggingFace. > And How am I supposed to know that arriving to huggingface as a new user? Enlighten me. Because I told you, knowing it was the best starting point for newbies. > Cool.. Why don't they share em because I genuinely cant find em, I'm dumb. You…

[deleted]

Re: Qwen 3.8 27B

#678

Post by bilekas »

MrDrMcCoy wrote: post #625
bilekas wrote: post #618

Earlier quoted context omitted.

> I went to Huggingface, went to the Unsloth org, as they tend to be the best, went to the Model page, and went to the "Files and versions" tab. "I went to youtube, gmail, ycombinator, deliveroo, then I went to another site I randomly chose, because they're the best, duh" Okay. > Unsloth AI is a very popular, highly reputable organization that takes upstream model files And How am I supposed to know that arriving to…

> "I went to youtube, gmail, ycombinator, deliveroo, then I went to another site I randomly chose, because they're the best, duh" The path I described is all within HuggingFace. > And How am I supposed to know that arriving to huggingface as a new user? Enlighten me. Because I told you, knowing it was the best starting point for newbies. > Cool.. Why don't they share em because I genuinely cant find em, I'm dumb. You…

> The path I described is all within HuggingFace.

It's just not. Where do I see the model download ?

The fact we're here is a loss.

It's not intuitive. Deal with it, or fix it.

Re: Qwen 3.8 27B

#679

Post by z4y5f3 »

skrebbel wrote: post #178
satvikpendem wrote: post #161

As usual, the Jinja templates are messed up so use this [0] to reduce or turn off thinking, fix tool calling, keep a 100% KV cache hit rate, etc. [0] https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

I'd love to understand this more. Are you saying the Qwen team spends their very impressive human and compute resources on publishing these amazing models and then botches the chat template with mundane bugs? Like maybe I just misunderstand what's the hard part but wouldn't you assume that people who can put together an impressive model can also write a proper jinja chat template for it?

Yes, and this is not the first time they messed up. They had tokenizer bugs where the trained weights do not match the template back to Qwen3 series.

Re: Qwen 3.8 27B

#680

Post by pulse7 »

CMay wrote: post #518

Credit where it's due. Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. It took 5x as many tokens to do it and 12m30s with MTP enabled, but it did do it. Gemma 4 reasoned through it more implicitly, while Qwen 3.8 reasoned more explicitly. Laguna and Muse Glimmer failed hard on it, though they're useful for other tasks. The VRAM usage see…

Can you please tell which Gemma 4 variant managed to correctly reason through your private benchmarks? Was is Gemma 4 31B?

What quantizations and context lengths did you use for Gemma 4 and Qwen 3.8 27B?

I am asking because I can't even load Gemma 4 31B on my GPU with any reasonable quantization (even with small context), while I can run Qwen 3.8 27B with large context and good quantization...

Post reply on HN