Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

31–40 of 848 posts

Re: Qwen 3.8 27B

#31

Post by WithinReason »

scrlk wrote: post #8

Beats Opus 4.7 Max (w/ Claude Code) on DeepSWE (42.2 vs 40). Looks like Qwen's 27B models continue to pack some punch. Unsloth's GGUF quants are up: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

I wish each quant was benchmarked on the same tests as the original network so we could compare their performance

Re: Qwen 3.8 27B

#32

Post by brcmthrowaway »

TomGarden wrote: post #24

Any tips on the best approach at running this at an M4 Max 128GB? Token throughput was a bit slow with the last 27B one (MLX), ended up using the A3B variant but if I could get this one to reasonable speed I'd much prefer it.

Check out MTPLX and limit your context size.

Re: Qwen 3.8 27B

#34

Post by NitpickLawyer »

scrlk wrote: post #8

Beats Opus 4.7 Max (w/ Claude Code) on DeepSWE (42.2 vs 40). Looks like Qwen's 27B models continue to pack some punch. Unsloth's GGUF quants are up: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

> Beats Opus 4.7 Max

I'm a huge open model fan, and have used them since forever, even have daily drivers for on-prem dev, but no. They do not beat opus on real-world usage.

Qwen models are impressively good for what they are, are "good enough" for plenty tasks, can be ran locally on decently priced hardware, and so on. They certainly have their uses, and the field in general has advanced faster than my early expectations. But to compare a 27B model to SotA behemoths from a few months ago is doing everyone a disservice, especially people who pick it up, try to use them just like API models, and leave disappointed and confused. Number goes up on a benchmark isn't it.

Re: Qwen 3.8 27B

#37

Post by petu »

nblgbg wrote: post #14
scrlk wrote: post #8

Beats Opus 4.7 Max (w/ Claude Code) on DeepSWE (42.2 vs 40). Looks like Qwen's 27B models continue to pack some punch. Unsloth's GGUF quants are up: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

Is there any advantage to using the model from Unsloth compared with https://huggingface.co/Qwen/Qwen3.8-27B-FP8 ?

Unsloth one is gguf for llama.cpp (and some other on-device engines).

So advantage is not having to produce your own quantisation / gguf from .safetensors you've linked.

Re: Qwen 3.8 27B

#39

Post by mft_ »

TomGarden wrote: post #24

Any tips on the best approach at running this at an M4 Max 128GB? Token throughput was a bit slow with the last 27B one (MLX), ended up using the A3B variant but if I could get this one to reasonable speed I'd much prefer it.

Go for a slightly more quantised version, and experiment with different MTP settings. I find that MLX versions are marginally faster on my 64GB M1 Max, but I usually use Unsloth's GGUFs via llama.cpp as there's a much greater range of quants available and I prefer llama.cpp. MTP sometimes also helps a little, but I suspect it's less helpful on my system than others.

Unsloth: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

This might work for you, but I didn't get on very well with MTPLX when I tried it a while back; YMMV: https://huggingface.co/Youssofal/Qwen3.8-27B-MTPLX-Optimized...

Re: Qwen 3.8 27B

#40

Post by 4chandaily »

nblgbg wrote: post #14
scrlk wrote: post #8

Beats Opus 4.7 Max (w/ Claude Code) on DeepSWE (42.2 vs 40). Looks like Qwen's 27B models continue to pack some punch. Unsloth's GGUF quants are up: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF

Is there any advantage to using the model from Unsloth compared with https://huggingface.co/Qwen/Qwen3.8-27B-FP8 ?

Run the unsloth if you are using llama.cpp (GGUF)

Run the one you linked if you are running vllm (safetensors)

Post reply on HN