https://news.ycombinator.com/item?id=48403639
my prediction was way too far out. 4.6 at home! Woo.
Qwen 3.8 27B
71–80 of 848 posts
Re: Qwen 3.8 27B
#72Unsloth Q4_K_M on a single 3090, llama.cpp
"Generate an SVG of a pelican riding a bicycle" first try
https://www.reddit.com/r/LocalLLaMA/comments/1voa3ch/comment...
Re: Qwen 3.8 27B
#73Earlier quoted context omitted.
> I've settled on GLM-5.3 (formerly Deepseek v4 pro 0813) for architecting Dude, GLM-5.3 released _today_. The phrasing "I've settled on" is incorrect for this context.
hence the "former deepseek v4 pro". I tried it out this morning and have had no complaints. I already liked glm 5.2
The sentence still doesn't make sense, because "settled on" implies a long testing phase with a verdict eventually emerging out of that.
What you're currently doing is "testing out"
Re: Qwen 3.8 27B
#74Beats Opus 4.7 Max (w/ Claude Code) on DeepSWE (42.2 vs 40). Looks like Qwen's 27B models continue to pack some punch. Unsloth's GGUF quants are up: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF
I wish each quant was benchmarked on the same tests as the original network so we could compare their performance
Unsloth publishes KL divergence numbers which measures how much the quantised probability distribution changes vs unquantised: https://unsloth.ai/docs/models/qwen3.8#quantization-analysis
It's a bit bare at the moment, I assume they are going to add further detail later (eg comparison to other quants), similar to their other releases.
Re: Qwen 3.8 27B
#7527b dense model at Opus 4.6 level Opus at home I hope there also will be a new ~10b variant
can you tell me ideas of usecases of 9 or 10B language models ? I cant find any usecases other than training a lora on them to give good bash commands for example
they are all overlapping but:
categorization, information retrieval, semantic search, image description
also with the model as part of an agentic system with tool calling
(edit: it is quite impressive what a small model in a feedback loop can do)
Re: Qwen 3.8 27B
#76Note: on the model card the comparison to Opus is Opus 4.6 Max, not 4.7
Re: Qwen 3.8 27B
#77Any tips on the best approach at running this at an M4 Max 128GB? Token throughput was a bit slow with the last 27B one (MLX), ended up using the A3B variant but if I could get this one to reasonable speed I'd much prefer it.
Go for a slightly more quantised version, and experiment with different MTP settings. I find that MLX versions are marginally faster on my 64GB M1 Max, but I usually use Unsloth's GGUFs via llama.cpp as there's a much greater range of quants available and I prefer llama.cpp. MTP sometimes also helps a little, but I suspect it's less helpful on my system than others. Unsloth: https://huggingface.co/unsloth/Qwen3.8-27B…
This is the way if you need speed. It costs a little bit in smarts, but compare the MTPLX option listed above with the oQ4e-mtp quant using oMLX. The good cacheing layer in oMLX will help things feel faster for some classes of tasks in my experience.
Re: Qwen 3.8 27B
#78Beats Opus 4.7 Max (w/ Claude Code) on DeepSWE (42.2 vs 40). Looks like Qwen's 27B models continue to pack some punch. Unsloth's GGUF quants are up: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF
Good morning Dario!
Re: Qwen 3.8 27B
#79Beats Opus 4.7 Max (w/ Claude Code) on DeepSWE (42.2 vs 40). Looks like Qwen's 27B models continue to pack some punch. Unsloth's GGUF quants are up: https://huggingface.co/unsloth/Qwen3.8-27B-GGUF
Considering the clusterfuck that is opus 5 or even fable, if Qwen 27B is trully better than Opus 4.7 Max, I will rejoice.
If it's as good as Sonnet 4.6 for most things I'd be happy.
Re: Qwen 3.8 27B
#80Earlier quoted context omitted.
> Beats Opus 4.7 Max I'm a huge open model fan, and have used them since forever, even have daily drivers for on-prem dev, but no. They do not beat opus on real-world usage. Qwen models are impressively good for what they are, are "good enough" for plenty tasks, can be ran locally on decently priced hardware, and so on. They certainly have their uses, and the field in general has advanced faster than my early expecta…
> ...but no. They do not beat opus on real-world usage. I agree, but then we just need meaningful benchmarks that clearly show that! Otherwise it's hand waving about something that should be put on paper in quantifiable terms.
If you are working in a company and using language models, it is a very good idea to hold a bunch of evals you can trust and use to validate new models. Calibrate every once in a while with prod data. We have our own and the only numbers on quality and cost I trust come from this setup.