Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

541–550 of 848 posts

Re: Qwen 3.8 27B

#541

Post by MrDrMcCoy »

bilekas wrote: post #479
MrDrMcCoy wrote: post #458

Earlier quoted context omitted.

Official llama.cpp releases ship with huggingface support. If you don't want to download it yourself, you can just use the `repo/model:quant` convention and it will handle downloading locally for you.

But you're assuming I'm using the Olama studio. This model as far as I see doesn't have a gguf download.. Unless I'm missing something on the page. If I want to download the model myself, it's not clear. I thought it was supposed to behave like a package manager. But even in nuGet I can download a zip of the package.

What on earth are you talking about? llama.cpp != Ollama. You can (and should) just use llama.cpp directly. Upstream llama.cpp can take the shorthand huggingface path and automagically download it into a cache folder as part of the launch. Have you read any of the docs?

Re: Qwen 3.8 27B

#542

Post by try-working »

kzrdude wrote: post #398
isomorphic wrote: post #367

Earlier quoted context omitted.

"inclined to overthinking," holy cats you're not kidding! On a Mac mini M4 Pro 64GB I prompted it with "svg owl" and it thought for 17m12s, outputting 36.3KiB of thinking chatter. It did end up producing a 20.2KiB HTML+JS+SVG file with a very nice owl, including cursor-tracking animation, but it ran for more than a half hour! The MoE models are stupendously faster.

I think that's called test-time scaling i.e using more tokens at infer time to squeeze out higher model performance. That's must be part of the explanation for good benchmark results.

that seems to be how most Chinese models achieve increased benchmark scores. GLM and Kimi models are "thinkslop" models that reason over their own thinking, which increases cost and decreases speed significantly. That's why GPT is in a different tier altogether - faster, smarter, and sometimes cheaper.

Re: Qwen 3.8 27B

#543

Post by MrDrMcCoy »

bilekas wrote: post #482
MrDrMcCoy wrote: post #458

Earlier quoted context omitted.

Official llama.cpp releases ship with huggingface support. If you don't want to download it yourself, you can just use the `repo/model:quant` convention and it will handle downloading locally for you.

Plain question for you, where can I find the gguf model of this to direct download ?

Here: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/tree/main

Re: Qwen 3.8 27B

#544

Post by walrus01 »

dofm wrote: post #417

There's a real change (compared to 3.6) in the way it writes in thinking — it drops words like "to" and "we" in "We need to", talks generally in note form, drops the/and all over the place, avoids "for". "Need be helpful concise", "Need maybe not overdo", "Need ask!" Almost caveman. I have an (unsourced, vague) suspicion that this rather unique thinking trace pattern is actually hobbling the MTP predictions, which se…

I also noticed that if you ask it to write text content to a certain word length (like "please write 250 words about the potato"), it now takes its sweet time "manually" counting each word in its draft output. It puts a number next to each word and iterates through all of its draft paragraphs and tries to stick to a strict word count. This is with temperature 1.0 as recommended.

Re: Qwen 3.8 27B

#545

Post by bigyabai »

literoldolphin wrote: post #289
ferrouswheel wrote: post #268

Earlier quoted context omitted.

Lol, "burn money on apple hardware instead!"

And yet it's also a laptop you can basically take anywhere unlike a giant video card with 1000 watt power supply requirements.

Nvidia and AMD both have their own unified memory laptop SOCs, now. Apple Silicon's GPU is relatively weak, it's one of the less-efficient ways to use 100w for compute.

Even the fastest Apple Silicon chips like the M5 Max and the M3 Ultra still put up worse GPU compute performance than last-gen laptop RTX 4080 chips. And they don't scale, the largest M3 Ultra cluster you can configure is still ~2,000x smaller than a DGX SuperPOD. There's a reason Apple discontinued their rackmount hardware, there's very little demand for Apple Silicon in the datacenter.

Re: Qwen 3.8 27B

#546

Post by bilekas »

MrDrMcCoy wrote: post #543
bilekas wrote: post #482

Earlier quoted context omitted.

Plain question for you, where can I find the gguf model of this to direct download ?

Here: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF/tree/main

Excuse me, but thats a direct link you've just sent. I asked where I can find the links. I like to believe in the source of truth.

They shared a lot of links, I'm struggling to find yours. Where did yours come from ?

Who is Unsloth AI? Have they modified the model ? Is this really the source of truth ?

Do you see how steep the barrier for entry is to do anything right ?

unslothai is not a name qwen has ever used. So you're sharing a link to a model that isn't from the owner, while saying it's the owner's. I'm not comfortable with that, and I want AI to be a better tool.

Re: Qwen 3.8 27B

#547

Post by rootusrootus »

altruios wrote: post #432
jLaForest wrote: post #317

Earlier quoted context omitted.

i care about not financially supporting a person that is actively trying to disenfranchise me, why is that a difficult concept for some people? that not everyone is motivated exclusively by financial profit? is moral bankruptcy so pervasive that some people assume it is unanimous?

'They' - in this case - are those who are unable to empathize. People who only learn of other's struggles through visceral personal experience. Conservatives, in general, struggle with this class reasoning... Very much a 'if I don't experience it then it doesn't exist' kind of mentality. Frustrating and exhausting. A lack of imagination, a lack of empathy, they just are lacking in these regards.

Your comments are all in absolutes. As if this was the one thing that mattered to you, everything else is irrelevant. Don't most people make their decisions based on a balance of considerations? Dismissing half the world as nazi or fascist does not seem productive, it kinda makes you look like the insane one.

Re: Qwen 3.8 27B

#548

Post by searealist »

peri-cl wrote: post #322
peri-cl wrote: post #27

Earlier quoted context omitted.

Same here! Qwen3.6-35B-A3B is the only local model I've found that runs reasonably on my iGPU. Looks like me and and my noisily-wheezing laptop will be sitting out this upgrade.

[self-reply because comment edit window ended]: I now have a head-to-head benchmark. On my wheezy laptop (specs in sibling comment), this new dense model, Qwen 3.8 27B, gets ~4 tokens/second on generation. The older mixture-of-experts model, Qwen 3.6-35B-A3B, gets ~20 tokens/s. MoE is literally 5x times faster (on CPU) than comparable dense Qwen! Tested: Qwen3.6-35B-A3B-UD-Q4_K_XL Qwen3.8-27B-UD-Q4_K_XL

If you are not already using MTP, you should be able to get ~2x decode tokens/s with Qwen 3.8 27B.

Re: Qwen 3.8 27B

#549

Post by hannofcart »

culi wrote: post #531
hannofcart wrote: post #525

Earlier quoted context omitted.

Another way of rephrasing that though is that the public (including large pension funds, retirees, etc.) just dodged a bullet and aren't left holding the bag. Nevertheless, I doubt anyone's going to emerge unscathed when the valuations start falling though. Vast sections of the American stock market will be wiped out yet and with it the global economy.

Why do you assume 2-3 American companies failing will have a more negative impact on the economy than positive impact of the widespread availability of self-hostable, open-sourced, and efficient models? There will surely be tons of new companies that make a business off of hosting these models and even tuning them for specific purposes. Not to mention the massive pricing difference will benefit people who actually ut…

Firstly, I don't think it will be just 2-3 companies that fail. Due the circular investing between companies in the AI ecosystem, the fallout is going to be much bigger.

Secondly, even with that being true, I think open models are a great net benefit for all mankind in the long run.

I think both of those things are true at the same time.

Re: Qwen 3.8 27B

#550

Post by dofm »

CMay wrote: post #518

Credit where it's due. Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. It took 5x as many tokens to do it and 12m30s with MTP enabled, but it did do it. Gemma 4 reasoned through it more implicitly, while Qwen 3.8 reasoned more explicitly. Laguna and Muse Glimmer failed hard on it, though they're useful for other tasks. The VRAM usage see…

I was quite impressed by Muse Glimmer, and while I am sure people will observe that it is less good on benchmarks, my first experiences with this new 27B have been somewhat exasperating, whereas testing Muse Glimmer was rather fun. I have not tested either in an agentic context, mind you.
Post reply on HN