Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

modelscope.cn

Post reply on HN
Print view View original

131–140 of 178 posts

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#131

Post by FuckButtons »

wongarsu wrote: post #68
cpburns2009 wrote: post #34

Earlier quoted context omitted.

So back in the Qwen 3.5 release, the 122B-A10B model scored slightly better than the 27B model. I'd expect this new 125B-A6B to perform similarly to the recently released 27B. Qwen3.8 27B is supposed to rival Sonnet/Opus 4.6.

There is the rule of thumb that if you take the geometric mean of the total and active parameters of an MoE model you get the equivalent size of an equally capable dense model. If you follow that formula, you would expect a 125b-a6b model to match a 27b model (sqrt(125*6) = 27.3). That does not feel like a coincidence

Where does that rule of thumb come from?

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#132

Post by DwarvenEngineer »

brandall10 wrote: post #127
embedding-shape wrote: post #98

Earlier quoted context omitted.

And the newly announced/launched Apple M6 has 170GB/s of unified memory bandwidth, meanwhile M5 Ultra gets 1.2TB/s of unified memory bandwidth. https://www.apple.com/newsroom/2026/08/apple-introduces-m6-a... Not sure if the first one is a typo on their press release, can't be just 170GB/s then be pushed for AI use, can it? Could be a different measurement I suppose...

From a bandwidth perspective, the ultra is like 8 M5s fused together (@ 150GB/s), that's how it gets to the 1200. Historically the Pro doubles the base, the Max doubles the Pro, and the Ultra doubles the Max. If an M6 ultra were released today it would be 1.36TB/s.

does that mean they're measuring bandwidth differently than how others (like nvidia) does it? memory bandwidth is the gating factor of running models locally, so if it's actually 8x 150GB/s, it may help something like prefill, but would it actually speed up decode comparatively?

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#135

Post by cpburns2009 »

vorticalbox wrote: post #104
WiSaGaN wrote: post #85

Earlier quoted context omitted.

You probably meant Qwen3.8-35B-A3B. But judging from some of the words from their team, it seems unlikely unfortunately.

They normally release a 35b dense and an 27b moe (4B active per token) For context 35B on my m4 runs at 10 tokens a second, 27B moe runs 50-60 tokens a second.

You have your numbers switched. 27B is the dense model and runs slowly on unified memory. 35B (A3B active) runs great on unified memory.

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#136

Post by copperx »

irthomasthomas wrote: post #17
ddtaylor wrote: post #14

I enjoy the Qwen models a lot, but building things on top of them with OpenRouter has been painful. OpenRouter does a lot of great work and I really enjoy being able to use different models so easily. I like when a provider is phasing out an older model that still works for my needs and the price is much lower. It seems like such a good win-win. However, the problem is that many Qwen models have almost no capacity or…

Openrouter was pretty great before prompt caching became common. Now it is extremely expensive for most individual workflows, unless you spend a lot of work customizing router preferences, and then you still get a worse cache hit rate than using the provider directly. I only keep $5-$10 in OR for occasional testing.

Why is caching affected when not using the provider directly?

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#137

Post by rdsubhas »

pwython wrote: post #11

I was already rolling around the idea of a 128GB M5 Max MBP. Now this! A 4-bit MLX quant with 128k window should fit perfectly, in the 50-70 tok/s range.

How do you folks code at 40-50 tps? With an extremely lightweight harness (pi) and just 8k system and tools context, and ~40tps on qwen 3.8 27B 4-bit on low thinking mode, it still takes me nearly 30-45 mins for a basic coding session...

Does it work? yeah... But I'd pick a subscription anyday...

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#138

Post by smcleod »

Casteil wrote: post #38
smcleod wrote: post #20

Earlier quoted context omitted.

50-70tk/s is what I get on my m5 max on a 5-6bit Qwen 3.8 27B?

I don't know what black magic you're up to but I see more like 30-35t/s on a 16" M5 Max using 3.8:27b Q4, regardless of whether it's mlx or gguf. qwen3.5:122b-a10b is significantly faster at around 60-65.

No magic, just oMLX with MTP. You can look through the speed the community is getting here: https://omlx.ai/benchmarks/performance?model=qwen3.8&chip=&c...

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#139

Post by decide1000 »

SwellJoe wrote: post #82
throwaw12 wrote: post #67

Earlier quoted context omitted.

how much performance (tok/s) can you expect from 128GB Strix Halo? assuming this model will be released with FP8 also can you use it for fine tuning?

The Strix Halo and DGX Spark are pretty danged slow, relatively speaking. I don't recall exact numbers, but with MoE models in this size ballpark (Laguna S 2.1), I seem to recall I was seeing about 20-25 t/s with a big context, which is close to usable. Qwen 3.8 27B crawls on this hardware, though, at 10-16 t/s, definitely not comfortable for interactive use. (Though this makes it seem like you can cook pretty good w…

On the DGX I get 44.5 tokens per second (NVFP4). With 8 concurrent it's 241 t/s total.

I am using the PrismaAQUA

standard 9.7 t/s

+ Dflash2 30 t/s

+ torch-compile 37 t/s

c8 = 177 t/s

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#140

Post by SwellJoe »

decide1000 wrote: post #139
SwellJoe wrote: post #82

Earlier quoted context omitted.

The Strix Halo and DGX Spark are pretty danged slow, relatively speaking. I don't recall exact numbers, but with MoE models in this size ballpark (Laguna S 2.1), I seem to recall I was seeing about 20-25 t/s with a big context, which is close to usable. Qwen 3.8 27B crawls on this hardware, though, at 10-16 t/s, definitely not comfortable for interactive use. (Though this makes it seem like you can cook pretty good w…

On the DGX I get 44.5 tokens per second (NVFP4). With 8 concurrent it's 241 t/s total. I am using the PrismaAQUA standard 9.7 t/s + Dflash2 30 t/s + torch-compile 37 t/s c8 = 177 t/s

What model? Also, I don't know what "the PrismaAQUA" means, ddg thinks it's a CPAP machine, which seems unlikely to help with inference performance.

Also, 4-bit has measurable intelligence loss. Sometimes worth it, but, at this size models are barely smart enough at 8 or 6.

Post reply on HN