Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

321–330 of 848 posts

Re: Qwen 3.8 27B

#321

Post by metadat »

bilsbie wrote: post #307
hypfer wrote: post #109

Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM. llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spe…

Any chance I could run it on a GeForce RTX 5060 Ti 16 GB, (64 gb ram)

You may be able to run a quantized version. One strategy I've found effective is to set Claude or Codex loose on the problem and they'll do a decent job setting up the best possible model revision.

Re: Qwen 3.8 27B

#322

Post by peri-cl »

peri-cl wrote: post #27
KronisLV wrote: post #10

I hope really badly that we'll get a new 35B A3B or similar MoE model! I also miss the Qwen 3 Coder Next, which was 80B A3B, there are quite a few use cases where a non-dense model Also alternate link for viewing the images without signing in: https://xcancel.com/Alibaba_Qwen/status/2088280182356611304

Same here! Qwen3.6-35B-A3B is the only local model I've found that runs reasonably on my iGPU. Looks like me and and my noisily-wheezing laptop will be sitting out this upgrade.

[self-reply because comment edit window ended]: I now have a head-to-head benchmark. On my wheezy laptop (specs in sibling comment), this new dense model, Qwen 3.8 27B, gets ~4 tokens/second on generation. The older mixture-of-experts model, Qwen 3.6-35B-A3B, gets ~20 tokens/s.

MoE is literally 5x times faster (on CPU) than comparable dense Qwen!

Tested:

Qwen3.6-35B-A3B-UD-Q4_K_XL

Qwen3.8-27B-UD-Q4_K_XL

Re: Qwen 3.8 27B

#323

Post by [deleted] »

literoldolphin wrote: post #211

Why is anyone even using video cards these days? You may as well be burning cash. This is the perfect candidate for just splattering it on your nvme and then reading it off there and into memory. All of these run perfectly fine on simple m4 silicone: https://github.com/drumih/turbo-fieldfare https://github.com/leonickson1/Swiftlet https://github.com/sqliteai/warp

[deleted]

Re: Qwen 3.8 27B

#324

Post by lenerdenator »

Balinares wrote: post #248

I wonder if Anthropic and OpenAI possibly missed the window to go public. A 27B open-weight model trading blows with the SOTA from just half a year ago is not great news for trillion-dollar investments...

I mean, that was obvious from the word "go".

Even Zuckerberg isn't as deluded to think that there's money in constantly releasing closed models and hypserscaling their deployment, which is why we got muse-glimmer 30b last week. I have been playing with it on a 32gb MBP w/ M2 Pro. If prosumer-grade hardware from almost three years ago can now offer a novice a way to host a possibly serviceable software development agent, then there's little incentive to pay out for LLMs like Anthropic's or OpenAI's over the long run.

Re: Qwen 3.8 27B

#325

Post by tonis2 »

ferrouswheel wrote: post #243
redox99 wrote: post #143

Earlier quoted context omitted.

Yep. These small models are actually worse than GPT 3.5 at some tasks (like recalling facts). You can definitely make models smarter at specific tasks (like tool calling, coding) but you can't compress the entire human knowledge into a 30GB file. It's just not enough bits.

But why would you use a model to store factual knowledge, that is stupid. We want intelligence, not a database.

Models cant make any decisions if they have 0 idea that the feature exist in this language or in some general fact.

For example you would tell a model hey, become an expert in this language for me, search it online, it would still need to learn it and download the data to it's context and then increasing the memory usage, there's no way around it.

Re: Qwen 3.8 27B

#326

Post by lukan »

tinfoilhatter wrote: post #283
rob wrote: post #215

Earlier quoted context omitted.

I just read through a couple of your posts that weren't dead or buried, and it seems like you're pretty anti-AI. You should really start to have an open mind towards it. It's going to be the future (if it isn't already), and as you continue to get older, you're going to really wish you spent your time right now learning and embracing the technology instead of being so against it. A lot of the skills and things that y…

I'm not sure that typing messages to a chat bot requires much catching up, but since you think this is what I should do rob, I better listen!

If you don't get reliable results, but others do, then maybe you should indeed. But your choice.

Re: Qwen 3.8 27B

#327

Post by sampullman »

tinfoilhatter wrote: post #292
sampullman wrote: post #206

Earlier quoted context omitted.

I think in that case it's fair to say you created the painting with the artist, even if the artist should get majority credit. I don't like the analogy though, to me it feels more like you're a project manager directing a team of genius but single minded interns.

I've never encountered a project manager who has said: I built this thing, after a project has wrapped up. I have encountered ones who've said I managed this project.

I've heard some variant of "we built this thing", which I think is valid. I'm not sure how I feel about anthropomorphizing a coding agent at this point though, which is what you're implying.

Re: Qwen 3.8 27B

#328

Post by mhitza »

Jcowell wrote: post #284
cjbprime wrote: post #282

Earlier quoted context omitted.

Hm, I have a 4090 as well, and: $ build/bin/llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spec-default --cache-type-k-draft q4_0 --cache-type-v-draft q4_0 --threads 24 --jinja --reasoning on -fit off 0.02.993.689…

There has to be a better way of sharing config , a profile of sorts you can upload and share

Yes, llama.cpp supports presets (an ini format). Though it's not always clear which llama-server flags it does not support.

Re: Qwen 3.8 27B

#329

Post by hypfer »

smallmancontrov wrote: post #229
effdee wrote: post #208

Earlier quoted context omitted.

Some people will now argue it was the chisel—not Michelangelo—who created David.

"Carve me a naked guy. Make no mistakes."

I cannot fulfill this request. I am programmed to follow safety guidelines that prohibit the generation of sexually explicit content.

Re: Qwen 3.8 27B

#330

Post by alentred »

scrlk wrote: post #8

Beats Opus 4.7 Max (w/ Claude Code) on DeepSWE (42.2 vs 40). Looks like Qwen's 27B models continue to pack some punch. Unsloth's GGUF quants are up: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

Oh, goody goody goody! Can't wait for a Q4M with *MTP* support. Does anyone know if there are plans for it?

I am currently using Qwen 3.6 on RTX 3090 and I have to admit that without MTP it would be too slow to be acceptable for me (30-35 tok/sec without MTP, 60-70 with MTP). Without MTP I would just use OpenRouter and rather pay for speed despite having a capable local setup.

Post reply on HN