Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

701–710 of 848 posts

Re: Qwen 3.8 27B

#701

Post by CMay »

jodleif wrote: post #691
pulse7 wrote: post #680

Earlier quoted context omitted.

Can you please tell which Gemma 4 variant managed to correctly reason through your private benchmarks? Was is Gemma 4 31B? What quantizations and context lengths did you use for Gemma 4 and Qwen 3.8 27B? I am asking because I can't even load Gemma 4 31B on my GPU with any reasonable quantization (even with small context), while I can run Qwen 3.8 27B with large context and good quantization...

Then you might be missing SWA. Gemma models are extremely memory hungry without

So long as they have flash attention enabled, Llama.cpp enables Sliding Window Attention by default for Gemma 4 models. Even if they're using Ollama or LM Studio I would expect those to mostly be doing the right things.

Re: Qwen 3.8 27B

#702

Post by ziofill »

CMay wrote: post #657
urbnspacecowboy wrote: post #627

Earlier quoted context omitted.

> Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. I don't expect you to blab publicly about your private benchmark, but what sorts of reasoning does it require?

Well, I will say: #1: it does not require deep world knowledge, because that's not what local models are for. #2: it directly attacks drive-by understanding, overly linear processing training, poor attention mechanisms, poor reasoning patterns or lazy assumptions that ignore very easy low hanging fruit. #3: it requires solid instruction following in the face of errors. a lot of models will run into errors and then fa…

How much time did you invest in creating this benchmark? Any recommendations/resources you could give on how to do it?

Re: Qwen 3.8 27B

#704

Post by dexterlagan »

Tested the model briefly with my usual eval: a couple of questions on general knowledge most small models often get wrong, then write a fully-featured todo list app in JS, then rewrite the same app in Rust with Tauri. Granted, most models are well trained on basic todo apps, but it gives me an idea of the basic SWE capabilities I can build on. As far as I'm concerned, if it can successfully setup a local git repo, write a todo list app skeleton that works, I can work with it.

SWE: model is strong for its size. It one-shotted the Web app, had no bug. The Rust rewrite only had one bug (reordering didn't work immediately - fixed in one prompt). Committed locally then pushed to my GitHub (https://github.com/DexterLagan/RusTODO). Can't complain. If it can do that reliably, I can use it to make whatever I need.

General knowledge: I usually ask 2 questions many small models get wrong: summarize Operation Trojan Horse by John Keel, give publication year. Summarize the Ariel school incident of 1994. Qwen3.8 got the first question right, along correct publication year, gave glorious detail on Keel's theory, but got the second wrong. It thought that school was located in the USA. Ah well.

Performance on an admittedly overpowered laptop: 15 tokens/s in power save mode on this MacBook M5 Max 48GB, and 30 in performance mode. Perfectly usable for local coding through OpenCode.

Verdict: very nice local and free backup to my usual GPT/Claude/DeepSeek for code. Good for Web searches via Brave search tool calls. What more do you want from a small local model?

Re: Qwen 3.8 27B

#705

Post by nojs »

jjcm wrote: post #495
tomr75 wrote: post #415

Earlier quoted context omitted.

what token/s?

27 t/s. I suspect there will be significant speed ups in the coming weeks.

Any idea why it’s so slow? the entire model should fit in the vram of one card.

Re: Qwen 3.8 27B

#706

Post by Rayosay »

NamlchakKhandro wrote: post #592
lrvick wrote: post #87

Earlier quoted context omitted.

Use 3.6 27b as a daily driver for months with charmbracelet crush. Gemma 26b-A3b is not even remotely comparable in terms of coding for me. YMMV depending on how you work, what harness you use, etc I suppose.

really? crush.... it's trash harness compared to pi. I didn't realise there are people out there unironically using crush

What's wrong with it? I like Crush. It's hard to find good harnesses that don't pull in mounds of Javascript like Pi and OpenCode.

Re: Qwen 3.8 27B

#708

Post by terhechte »

dexterlagan wrote: post #704

Tested the model briefly with my usual eval: a couple of questions on general knowledge most small models often get wrong, then write a fully-featured todo list app in JS, then rewrite the same app in Rust with Tauri. Granted, most models are well trained on basic todo apps, but it gives me an idea of the basic SWE capabilities I can build on. As far as I'm concerned, if it can successfully setup a local git repo, wr…

Which setup did you use? MLX/GGUF, Quant, Engine (e.g. llama.cpp or MTPLX, etc)? There’s so much variety these days.

Re: Qwen 3.8 27B

#709

Post by jamesblonde »

For the full weights, unoptimized on vLLM with 2 Nvidia 6000 RTX 48GBs connected by NVLink, i only get 14 tokens/sec with open-code. For batched operations, it climbs to 55 tokens/sec.

For FP8, on a single Nvidia 6000 RTX 48GB, i get 13 tokens/sec on a single GPU and 46 tokens/sec batched.

Re: Qwen 3.8 27B

#710

Post by beltsazar »

simonw wrote: post #419

Absolutely the best pelican I've seen from a model that runs on my laptop: https://tools.simonwillison.net/markdown-svg-renderer#url=ht... Bicycle is the right shape. Pelican beak is excellent. Nice background. Most importantly, the pelican has one leg on each side of the bicycle - that's very rare. (No chain on this bicycle though - in the reasoning trace it says "already chainstay... skip chain detail; maybe a smal…

Using a better quant Q6_K (Unsloth's) compared to your quant Q4_K_M (LM Studio's) yields this: https://tools.simonwillison.net/markdown-svg-renderer#url=ht...

Chains exist. Red scarf is proactively added. ("Maybe a scarf blowing in the wind for charm!) No hands/wings, though.

Generated 30.2k tokens in total and took 52 mins on M5 Pro in low power mode (it will possibly take less than half of that in auto energy mode).

Post reply on HN