Live data from Hacker News

Qwen 3.8 27B

huggingface.co

831–840 of 848 posts

Re: Qwen 3.8 27B

#831
post #656

Earlier quoted context omitted.

> correctly reason through one of my private benchmarks Want to say more about these private benchmarks? :)

seems to me like "private" is a good descriptor - I also have a set of "private" test cases - and they are kept private on purpose so they aren't scraped and fine-tuned on.

I have a sneaking suspicion that Qwen is fine-tuned on youtuber test cases (like Luke's Dev Lab, where Qwen 3.8 27B has just done almost eerily well).

Part of my suspicion is drawn from the thinking trace I got when I tested the car wash problem. That really does seem to have been post-trained; it's too good.

e.g. Gemma 4 26B solves this concisely without adding any filler about fuel economy or how long it will take, but it generally gets there by breaking down the problem in the thinking trace the way you'd expect.

Qwen 3.8 27B is just a little too certain right off the bat in low reasoning mode.

Re: Qwen 3.8 27B

#832
post #296

I wonder if Anthropic and OpenAI possibly missed the window to go public. A 27B open-weight model trading blows with the SOTA from just half a year ago is not great news for trillion-dollar investments...

Trading blows in some benchmarks is a bit exaggerated. If you try the model, `xhigh` is basically feels like the `max` mode (i.e. massive thinker and extremely presistent), and the amount of world knowledge and intent understanding is nowhere close to an Opus class model even from 6mo ago. It's still very useful, and it'll probably displace a good bit of API spend; but it's not really "trading blows with SOTA from ju…

That's a reasonable point, so let me qualify: at a glance, it seems Qwen 3.8 27B can trade blows with Opus 4.6 on coding tasks, where trading blows doesn't necessarily mean it's a clear winner or even an equal, but does mean it'll at least hold its own and land a punch or two.

(Which I still think is bonkers, FWIW.)

Opus 4.6 is an especially interesting comparison point, I think, because it was a step change; IMO it's when LLMs became serviceable for coding. Yeah, pre-4.6 models did output code, and that code often superficially worked; and bringing it up to production standards still generally meant rewriting it entirely. Opus 4.6 is when that changed.

From my early tests, it's looking like the public benchmarks are not misleading, and Qwen 3.8 somehow got there too, by and large. I've got a few personal tests. One is a mid-complexity one-shot, purposefully underspecified. Beyond a few minor bugs that it could easily fix once pointed out, Qwen 3.8 largely aced it. There are a bunch of things I'd improve, but that was true of Opus 4.6's output too, and by and large the code is clean and well structured. Also worth noting that I'm running Qwen 3.8 fairly aggressively quantized to fit in VRAM; I'd expect tighter results still from the full weights.

Another test I ran is a variant of a common puzzle with an additional structural constraint that makes the usual solution inapplicable, so the model has to actively turn away from the well-known solution and construct a new one that takes the constraint into account. I've never seen a home model pass that test. Kimi K3 passes it, GLM 5.2 passes it (painstakingly). Qwen 3.8 struggles a lot... but does arrive at a correct solution. First time I see a home model do so.

I haven't yet tested it on long multi-turn scenarios. In my experience, that's where pocket models are weakest against heavyweight ones, especially when quantized.

That aside, it does seem like Qwen 3.8 can, in fact, trade blows with Opus 4.6. I don't know yet if it could replace it, and my money would be on no, but I may well be wrong about that considering how weirdly capable it is.

Interestingly, Qwen 3.8's MTP layer is uncannily accurate too. It still gave me good results up to 6 to 8 predicted tokens, which boosts its speed so much it's competitive with Qwen 3.6 MoE. So that's another bizarrely impressive thing about it.

And given all of the above, I do think that the trillions of dollars invested into OpenAI and Anthropic are becoming harder and harder to justify.

Re: Qwen 3.8 27B

#833

Earlier quoted context omitted.

> know that they don’t know And we are waiting for architectures that do - because it's duly.

Honestly it seems like a job for the harness, rather than the model. Sample the model with the same question, perhaps with varying temperature (?), and use that to establish a degree of confidence in the answer. If the model provides very different answers every time, respond that it doesn't know. If it responds with the same answer usually but a different answer sometimes, respond with moderate confidence. If the mo…

that could work sometimes but that's terribly hacky engineering

Re: Qwen 3.8 27B

#834
post #109

Since it might be helpful to some, here's my current commandline for llama.cpp running on an RTX 4090 with my monitor moved to the iGPU to free up all of its VRAM. llama-server -m Qwen3.8-27B-IQ4_NL.gguf --mmproj mmproj-BF16.gguf -c 170000 --parallel 1 -ngl -1 --cache-type-k q8_0 --cache-type-v q8_0 -b 1024 -ub 512 --flash-attn on --no-context-shift --no-mmproj-offload --spec-type draft-mtp --spec-draft-n-max 5 --spe…

Here's my model.ini for 4090 (using the chat template from froggeric):

version = 1

[*] ; Global defaults shared across all presets

threads = 16

[Qwen3.8-27B]

; Model files

model = ..\Qwen3.8-27B-FP8\Qwen3.8-27B-UD-Q4_K_XL.gguf

mmproj = ..\Qwen3.8-27B-FP8\mmproj-BF16.gguf

; Context & compute

ctx-size = 114688

parallel = 1

n-gpu-layers = -1

batch-size = 1024

ubatch-size = 512

flash-attn = on

cache-type-k = q8_0

cache-type-v = q8_0

; Speculative decoding (MTP draft)

spec-default = true

spec-type = draft-mtp

spec-draft-n-max = 5

cache-type-k-draft = q4_0

cache-type-v-draft = q4_0

; Multimodal

no-mmproj-offload = true

; Context behavior

no-context-shift = true

; Chat template & reasoning

jinja = true

reasoning = on

chat-template-kwargs = {"preserve_thinking":"true","reasoning_effort":"medium"}

; Sampling

temp = 1.0

top-n-sigma = 1

top-k = 0

top-p = 1.0

presence-penalty = 0.0

repeat-penalty = 1.0

fit = off

; Auto-load this model when the server starts

load-on-startup = true

Re: Qwen 3.8 27B

#835

Earlier quoted context omitted.

> know that they don’t know And we are waiting for architectures that do - because it's duly.

Honestly it seems like a job for the harness, rather than the model. Sample the model with the same question, perhaps with varying temperature (?), and use that to establish a degree of confidence in the answer. If the model provides very different answers every time, respond that it doesn't know. If it responds with the same answer usually but a different answer sometimes, respond with moderate confidence. If the mo…

If we want to implement Intelligence, and especially now that "the box is open" we must, we can play with the "intuitive" LLM architecture to understand it and squeeze it to its potential yeld, but at some stage we have to actually implement intelligence. That implies notions of confidence and a Foundational Theory of Knowledge (knowing why you know something), among the rest (one shot learning, update through reflection etc.).

Re: Qwen 3.8 27B

#836

Earlier quoted context omitted.

What is bpw? Also whats your cutoff for 'acceptable' speed? I would have said 25tok/s.

Bits per weight. I consider 'acceptable speed' to be around 150t/s. Why? Well, this is generally what it takes to keep me engaged with the output, rather than immediately switching to other tasks and checking back later. When I check back later, I have more catch-up to do at once, and I haven't been following the process. So I have to recall it, familiarize myself with the new progress, and sort of get back into focu…

Interesting, for anything more than side chats/projects I usually am watching the output generate and thinking about the problem. I have the same issue with switching back, takes a while to recall and page everything back into my context, so I try not to alt-tab away.

Re: Qwen 3.8 27B

#837
post #108

Earlier quoted context omitted.

Where do you see that? From what I can see on Open Router, Qwen 3.6 27B (the closest dense equivalent to Gemma 31) is $0.28/m. Am I missing something? https://openrouter.ai/qwen/qwen3.6-27b

They're comparing Qwen 3.8 Max to Gemma 31B, fundamental mistake.

[deleted]

Re: Qwen 3.8 27B

#838
post #108
post #57

Qwen 3.6 is ~$2/m tok, 3.8 should be drop in replacement. Gemma 31B is $0.34/m tok. The price differential on these models is massive on openrouter.

Where do you see that? From what I can see on Open Router, Qwen 3.6 27B (the closest dense equivalent to Gemma 31) is $0.28/m. Am I missing something? https://openrouter.ai/qwen/qwen3.6-27b

Your own link says:

In / Out Price

$0.289 / $2.40per 1M

Re: Qwen 3.8 27B

#839
post #518

Credit where it's due. Qwen 3.8 27B is only the second local model after Gemma 4 that managed to correctly reason through one of my private benchmarks. It took 5x as many tokens to do it and 12m30s with MTP enabled, but it did do it. Gemma 4 reasoned through it more implicitly, while Qwen 3.8 reasoned more explicitly. Laguna and Muse Glimmer failed hard on it, though they're useful for other tasks. The VRAM usage see…

this is efficiency im looking for. artificialanalysis.ai model review not up. so considering output token per intelligence, do u think is it better than muse glimmer or no?

Re: Qwen 3.8 27B

#840

Earlier quoted context omitted.

So add a pre-ingestion agent to your Pi Coding Agent subagent repertoire, problem solved.

Please, could you point me to any documentation about this pre-ingestion? I’ve looked here: https://github.com/earendil-works/pi/blob/main/packages/codi...

The point is to simply add a subagent that interprets your task and hands that off to the coder set to the appropriate level of thinking, essentially.
Post reply on HN