Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

341–350 of 848 posts

Re: Qwen 3.8 27B

#341

Post by onraglanroad »

tinfoilhatter wrote: post #276
mixologic wrote: post #236

Earlier quoted context omitted.

Nothing absurd about that. What do you think an "Executive producer" is? A "Director" ? Does Peter Jackson get credit for creating the Lord of the Rings Trilogy films? Christopher Nolan for his films? But did he make them ? No, it was the collective effort of thousands of individuals all working under their direction. Just like if somebody creates software today, and the end result is generated by the collective effo…

The director receives accreditation for directing the film, not creating it.

Doesn't the director generally receive more credit than the producer?

How many films do you remember the producer above the director?

Re: Qwen 3.8 27B

#342

Post by hedgehog »

ndriscoll wrote: post #320
hypfer wrote: post #313

Earlier quoted context omitted.

I wonder if we could take a page out of the Solar power book and do MPPT but for draft count. Constantly sweep through possible values to find the best result for the current conditions. Though it might turn out that that doesn't offer any benefits. Has it been tried yet? ___ EDIT: LLM pointed me towards this thing I'm not going to read https://arxiv.org/abs/2512.11280 "AdaSD: Adaptive Speculative Decoding for Effici…

Don't you just need to occasionally try n+1 for your current setting? Then record acceptance rates. The nature of the thing is that if you try n, you already have a good sampling of how well m works for all m<n, right?

This is all a pretty active area of research, both adaptive drafting, different draft approaches (DFlash etc), I expect to see really big improvements land in the popular runtimes this year.

Re: Qwen 3.8 27B

#343

Post by jhonof »

Yeah this is the first model I have been able to run locally that actually feels useful, this is unreal I am considering cancelling my claude sub and going to just api (maybe GLM?) for really hard tasks.

Re: Qwen 3.8 27B

#344

Post by venusenvy47 »

swalsh wrote: post #338

WOW, my first try running on my 2 3090's, it was a bit slow... but it FEELS like opus 4.5, i gave it an image and a broad overview of what I wanted it to build, and it built the whole thing from beginning to end.

For your setup, do you have both 3090's in parallel for the inference of the model?

Re: Qwen 3.8 27B

#345

Post by XCSme »

hypfer wrote: post #109

Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM. llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spe…

With default config via Ollama and 65k context I get 50tps on a 3090.

Re: Qwen 3.8 27B

#346

Post by SwellJoe »

bilsbie wrote: post #307
hypfer wrote: post #109

Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM. llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spe…

Any chance I could run it on a GeForce RTX 5060 Ti 16 GB, (64 gb ram)

You could run the 2-bit quantization at about 10GB for the weights. You'd probably also need to quantize context to get a decent-sized context. The Unsloth model card shows the 1-bit quantization doing seemingly shockingly well on a task (but there is notable loss in model quality for each bit you give up below about 6 bits). An MoE model can partially reside in system RAM without catastrophic performance loss, but a dense model like this one cannot. You have to run it entirely out of VRAM, or it'll slow to a crawl.

Re: Qwen 3.8 27B

#347

Post by seizethecheese »

Balinares wrote: post #248

I wonder if Anthropic and OpenAI possibly missed the window to go public. A 27B open-weight model trading blows with the SOTA from just half a year ago is not great news for trillion-dollar investments...

I don’t think so for two reasons:

1. They still have extreme revenue growth (and despite what HN says, good gross margins supposedly).

2. People will pay a premium for the best model. So far, we see catchup but no leapfrogging.

Re: Qwen 3.8 27B

#348

Post by XCSme »

swalsh wrote: post #338

WOW, my first try running on my 2 3090's, it was a bit slow... but it FEELS like opus 4.5, i gave it an image and a broad overview of what I wanted it to build, and it built the whole thing from beginning to end.

Why slow? I see ~50tps on a single 3090

Re: Qwen 3.8 27B

#349

Post by gered »

bogzz wrote: post #152
Almondsetat wrote: post #150

Earlier quoted context omitted.

Sorry, I wrote autocompletion by force of habit. I simply meant it can complete code you have already created a structure for, which personally is very nice

I thought so, but thanks for the clarification. I am a little bit disappointed that local autocompletion models have been left by the wayside in favor of models post-trained for agentic coding. Both Codestral and Qwen-2.5-coder are more than a year old at this point, but local auto-complete seems to me to be such a great usecase.

The latest Qwen models (including 3.8 27B) do still support FIM-style in-editor code auto-completion if that's what you're looking for. I wouldn't want to use a large dense model like 27B for such a task (since FIM-style auto-completion really works best with low-latency responses), but it works.

Re: Qwen 3.8 27B

#350

Post by graceful6800 »

enraged_camel wrote: post #202
spmurrayzzz wrote: post #48

Earlier quoted context omitted.

> They do not beat opus on real-world usage We have an internal eval that measures performance on tasks for a handful of embedded systems repos for our mmWave radios (mostly Rust, some C for microcontroller stuff). Qwen3.6-27B scores only 4% lower for pass@1, n=250 compared to Opus-4.8. For the labeled dataset, the average PR size they're being measured against is around 1.5k SLOC. This is very much "real-world usage…

[flagged]

They literally stated in their first sentence that it was coding tasks.
Post reply on HN