Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8 27B

huggingface.co

Post reply on HN
Print view View original

731–740 of 848 posts

Re: Qwen 3.8 27B

#731

Post by dexterlagan »

Yajirobe wrote: post #724
cjbprime wrote: post #714

Earlier quoted context omitted.

> General knowledge: I usually ask 2 questions many small models get wrong: summarize Operation Trojan Horse by John Keel, give publication year. Summarize the Ariel school incident of 1994. Qwen3.8 got the first question right, along correct publication year, gave glorious detail on Keel's theory, but got the second wrong. It thought that school was located in the USA. Ah well. I'm not sure that this means anything.…

Did the model refuse to answer? Did it say that it doesn't know? If not, then it's a fair game in my opinion.

Like most models it makes up things when it doesn't know. I had high hopes for Gemma4, which was said to having 'solved' this particular problem - it didn't. Gemma4 made less things up, did say it didn't know more, but it's still far from perfect. By comparison Qwen3.8 knows more, but still makes things up when it doesn't know. Coding abilities are very impressive however, and yeah these SVG tests do seem to 'scale' or generalize over its general reasoning+coding abilities - at least in JS and Rust. My next test will be to ask it to write some macros in Racket, just to see if it can balance parens. Most, if not all models cannot, no matter their size.

Re: Qwen 3.8 27B

#732

Post by harhargange »

jakswa wrote: post #587
spijdar wrote: post #464

I wonder how this practically compares with Muse Glimmer, especially quantized. I've got an RX 7900 XT (20GB of VRAM) and I can run glimmer with a full 128k context window with the draft model at 65-80 tok/s. This model, on the other hand, I get about 30 tok/s with a 30k context. Raising the context or loading the draft layers for MTP drops performance to 9-15 tok/s. So I wonder how big the "real world" delta between…

I'm in the exact same boat with a 7900 XT and a good Glimmer 30B experience. I was really hoping qwen 3.8 would bring some memory/space efficiency savings along the lines of whatever is going on with Glimmer 30B. I have been surprised that a 30 billion model fits and runs better (at higher unsloth quantization! UD-Q4_K_XL fits!) than a 27 billion model.

I too purchased the 7900xt as it was cheap with a lot of vram. Qwen 3.6 27b gives me 30 tok/s

Re: Qwen 3.8 27B

#734

Post by 127 »

johnnyApplePRNG wrote: post #565
farlight wrote: post #504

Earlier quoted context omitted.

They were significantly more expensive for coding compared to DeepSeek's offering because of much higher cost of cache reads and lower cache hit rate. Now, after the price jump, the effective cost of deepseek.com API vs 3-rd party providers is about the same.

DeepSeek's API was never "cheap". They were stealing your code for that extremely cheap cached token price. That was never the real price. Every Chinese LLM API provider is the same to the best of my knowledge. "CHEAP API" (but pssst we train on every single input aka your code and ideas are ours forever)

And you're claiming Anthropic and OpenAI are not doing the same thing?

Re: Qwen 3.8 27B

#735

Post by jacquesm »

jacquesm wrote: post #364
hypfer wrote: post #356

Earlier quoted context omitted.

Problem with that is I think that it quickly devolves into cargo culting, nonsense and noise. Arguably, what I am doing is also very very close to that, with the only difference being that I am somewhat less of an idiot than the average internet dweller you'd get on such a site. Or rather a different flavor of idiot. Ideally, the people building the tools build them in a way that just does the right thing - which I a…

I've found that once you factor in multiple GPUs things can get complex quite quickly because the default packing routine in the LLM runners tends to be very coarse resulting in substantial amounts of VRAM wasted. More so if you start running drafters and multiple models at the same time. Let me return the favor: Deepseek V4 Flash 0731 Q8 This is on a 4x3090 box with 256G RAM and a 5955WX TR; the CUDA drivers are pat…

The next morning: if you have long common prefixes then you may want to increase the --cache-ram setting.

Re: Qwen 3.8 27B

#736

Post by adrian_b »

literoldolphin wrote: post #683
bigyabai wrote: post #545

Earlier quoted context omitted.

Nvidia and AMD both have their own unified memory laptop SOCs, now. Apple Silicon's GPU is relatively weak, it's one of the less-efficient ways to use 100w for compute. Even the fastest Apple Silicon chips like the M5 Max and the M3 Ultra still put up worse GPU compute performance than last-gen laptop RTX 4080 chips. And they don't scale, the largest M3 Ultra cluster you can configure is still ~2,000x smaller than a…

As I mentioned above you can't take the data center into the cafe somewhere. We're talking about running local models here.

But you may be able to connect with your laptop to your home server, even from some cafe, and run the LLM remotely.

I always connect back home when I am away and I want access to a beefier computer. On the home server, "Wake on LAN" is enabled, so I can power it on and off from my home router.

Re: Qwen 3.8 27B

#737

Post by kzrdude »

dannyw wrote: post #555
kzrdude wrote: post #404

Earlier quoted context omitted.

Is the chat template used at all when they benchmark the model?

It’s unlikely they are benchmarking that downstream. They probably have private benchmark scripts with purpose-specific run telemetry and logging, etc. I’m sure there is some basic testing but it might be agentic (LLM likes its own output) and maybe just some human smoke tests.

So what's needed to solve this is that someone publicly and popularly benchmarks using the chat template. So that they have an incentive to look good using it.

Re: Qwen 3.8 27B

#739

Post by croes »

Yajirobe wrote: post #724
cjbprime wrote: post #714

Earlier quoted context omitted.

> General knowledge: I usually ask 2 questions many small models get wrong: summarize Operation Trojan Horse by John Keel, give publication year. Summarize the Ariel school incident of 1994. Qwen3.8 got the first question right, along correct publication year, gave glorious detail on Keel's theory, but got the second wrong. It thought that school was located in the USA. Ah well. I'm not sure that this means anything.…

Did the model refuse to answer? Did it say that it doesn't know? If not, then it's a fair game in my opinion.

Models don’t know that they don’t know.

Re: Qwen 3.8 27B

#740

Post by jchook »

WhyNotHugo wrote: post #718
dofm wrote: post #417

There's a real change (compared to 3.6) in the way it writes in thinking — it drops words like "to" and "we" in "We need to", talks generally in note form, drops the/and all over the place, avoids "for". "Need be helpful concise", "Need maybe not overdo", "Need ask!" Almost caveman. I have an (unsourced, vague) suspicion that this rather unique thinking trace pattern is actually hobbling the MTP predictions, which se…

This is pretty much Chinese grammar with English words. In Chinese you'd say "need ask" or "I need ask" if you want to be unusually verbose, but there's no "to". All your examples align with this too. It just sounds to me like its optimising by removing the superfluous English words and thinking more Chinesely.

After learning a little Mandarin I was amazed at how much simpler the Chinese grammar is than all Latin, Germanic, Romance, etc languages.

No gender, cases, tenses, conjugation, articles, word order inversion, etc. It's remarkable.

Mandarin has it's own complexities but grammar wise, it feels more optimized or simpler.

Post reply on HN