Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

581–590 of 848 posts

Re: Qwen 3.8 27B

#581

Post by eek2121 »

Casteil wrote: post #232
satvikpendem wrote: post #179

Earlier quoted context omitted.

Reduce or turn off thinking: https://huggingface.co/froggeric/Qwen-Fixed-Chat-Templates

Given that it apparently defaults to 'xhigh', this is probably the answer. Granted, it's still much lower tokens/s than you'll get out of many MoE models. Edit: Even set to medium or low there's still a lot of second guessing, less consistency, lower 'acceptable response' rate, and slower/more token churn vs gemma4:26b-a3b. I think gemma4 is just a better 'general purpose' model.

I haven't tried lowering thinking, however, I actually asked a solid question earlier regarding a real world scenario I encountered and all that excessive thinking made it give me an amazing answer. The thinking actually all made sense, and honestly I found it thought of similar stuff to what I thought when I drew my own conclusion.

I don't usually rely on AI for much (I'm actually kind of anti-AI, although I follow stuff like this enthusiastically because of the rapid advancements, "average joe" access, and openness), however for what I asked? It was spot on.

The subject was a bit personal, so I won't share. I was just curious what AI would say about the situation and it definitely surprised me, especially since Qwen, while usually great on development/coding stuff, has shown weaknesses in other areas.

Definitely a solid release, and this one runs on my 4090 with minimal loss of quality!

Re: Qwen 3.8 27B

#582

Post by bmitc »

Aurornis wrote: post #173
bmitc wrote: post #138

Earlier quoted context omitted.

Lol at that command. Why is this stuff so hard to run locally? I've spent a few days trying to figure it all out and haven't been able to. LM Studio doesn't work behind proxies. Ollama is confusing and doesn't seem to support Qwen3? And Llama.cpp is your command. I just want to run ` ` with some default parameters set and for it to run locally.

Start by copying the command line from the Unsloth guides. You don’t need to fine tune all of those parameters to get started. It’s really easy to ask an LLM to adjust the command line if you can’t be bothered to read the help out. Copy the help output into the LLM and tell it your goal. > Ollama is confusing and doesn't seem to support Qwen3? Typing “Ollama qwen3” into Google takes you right to this page: https://ol…

I had seen this:

> Attention: To be updated for Qwen3

on Qwen's official docs: https://qwen.readthedocs.io/en/latest/run_locally/ollama.htm.... It's not like I just made it up. Of course I searched "ollama qwen3" and saw what you linked, but that doesn't mean it "works". I have other things to do besides to try a bunch of poorly documented and executed tools just to see if it works or not.

I guess the TLDR is that I'm stupid or lazy. Also, everyone is responding about how easy it is, and yet, it's apparently so easy that it's hard to document well.

Re: Qwen 3.8 27B

#583

Post by eek2121 »

stusmall wrote: post #436
hypfer wrote: post #68

Earlier quoted context omitted.

> I've settled on GLM-5.3 (formerly Deepseek v4 pro 0813) for architecting Dude, GLM-5.3 released _today_. The phrasing "I've settled on" is incorrect for this context.

I'm generally extremely skeptical about a lot of the model hype that show up in comments. Except when there is an extreme mismatch the performance, quirks and quality of these things are difficult to nail down. You wouldn't know that from the comment section of every single release . I think some of these are excited, eager users always ready to hype up the new thing. The same crowd that previously would constantly p…

IMO this is getting hyped because the 27b version runs on a decent gaming GPU. This is NOT a thread for their largest model, this model will run on a mid-high end gaming PC, which you probably have in your household. Mine is 6 years old and it runs quite well.

Re: Qwen 3.8 27B

#585

Post by dofm »

ricardobeat wrote: post #558
dofm wrote: post #417

There's a real change (compared to 3.6) in the way it writes in thinking — it drops words like "to" and "we" in "We need to", talks generally in note form, drops the/and all over the place, avoids "for". "Need be helpful concise", "Need maybe not overdo", "Need ask!" Almost caveman. I have an (unsourced, vague) suspicion that this rather unique thinking trace pattern is actually hobbling the MTP predictions, which se…

This has been common in Kimi K3, and now DS Flash/Pro as well. Evidence of cross-training?

It looks like the caveman speak is a facet of the 'xhigh' reasoning effort. Set (bodged!) to medium it is not caveman.

Re: Qwen 3.8 27B

#586

Post by lta »

m_ke wrote: post #15
kristopolous wrote: post #9

q4km is about 48 tps on a 4090. my llama.cpp params are --flash-attn on --parallel 1 --load-mode mmap

With spec decode should easily get to >100tps on my dual 3090s qwen 3.5 27b was running at around 110tps using the config from https://github.com/noonghunna/club-3090 make that 200tps on a single 5090, 4x faster than opus https://x.com/radixark/status/2088285681131110446 devs about to get handed a two 5090 box each and told to max that out

I've been experimenting with a few settings in my 4090 , and if 3.6 run at 90-110 tps, 3.8 staya below 80 tps and it's most often at 60 tps. I'm using flash attention, mtp speculative decoding (n=2). I've looked at the club-3090 repo, but haven't found anything meaningful but get back to 3.6 performance

Re: Qwen 3.8 27B

#587

Post by jakswa »

spijdar wrote: post #464

I wonder how this practically compares with Muse Glimmer, especially quantized. I've got an RX 7900 XT (20GB of VRAM) and I can run glimmer with a full 128k context window with the draft model at 65-80 tok/s. This model, on the other hand, I get about 30 tok/s with a 30k context. Raising the context or loading the draft layers for MTP drops performance to 9-15 tok/s. So I wonder how big the "real world" delta between…

I'm in the exact same boat with a 7900 XT and a good Glimmer 30B experience. I was really hoping qwen 3.8 would bring some memory/space efficiency savings along the lines of whatever is going on with Glimmer 30B. I have been surprised that a 30 billion model fits and runs better (at higher unsloth quantization! UD-Q4_K_XL fits!) than a 27 billion model.

Re: Qwen 3.8 27B

#589

Post by wincy »

simonw wrote: post #419

Absolutely the best pelican I've seen from a model that runs on my laptop: https://tools.simonwillison.net/markdown-svg-renderer#url=ht... Bicycle is the right shape. Pelican beak is excellent. Nice background. Most importantly, the pelican has one leg on each side of the bicycle - that's very rare. (No chain on this bicycle though - in the reasoning trace it says "already chainstay... skip chain detail; maybe a smal…

Not too bad, on my 5090 I got ~160tokens/sec and it took 3m20s to generate a pelican using Ninfer which someone else mentioned. I don't know why I didn't get any sweet sweet seagulls in my background, even on xhigh reasoning effort.

https://gist.github.com/hansale/ed9e73fe35165a58ea2af6b1632a...

Re: Qwen 3.8 27B

#590

Post by NamlchakKhandro »

johnnyApplePRNG wrote: post #457
versteegen wrote: post #70

Earlier quoted context omitted.

IME using 5.6 Luna and DS V4 Flash, I notice that although they are excellent at programming, even Opus-like in the way they try to debug, the thing they are worst at is inferring user intent and making good decisions with little information. They are absolutely terrible at that, will misinterpret small wording ambiguities. I suspect that's an ability you can't add with RL training, that it requires the depth of unde…

So add a pre-ingestion agent to your Pi Coding Agent subagent repertoire, problem solved.

Woah woah woah buddy... suggesting that people use anything other than ClaudeCode or Codex is simply not allowed around these parts.
Post reply on HN