Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

modelscope.cn

Post reply on HN
Print view View original

31–40 of 178 posts

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#31

Post by try-working »

geek_at wrote: post #25
ddtaylor wrote: post #14

I enjoy the Qwen models a lot, but building things on top of them with OpenRouter has been painful. OpenRouter does a lot of great work and I really enjoy being able to use different models so easily. I like when a provider is phasing out an older model that still works for my needs and the price is much lower. It seems like such a good win-win. However, the problem is that many Qwen models have almost no capacity or…

The best solution to this for me is to self host litellm or a different router and use model aliases. For example I have a model called "coding" and when a new good model comes out I just switch the backend without needing to change the alias or the key in my projects (opencode, etc). I have a few of them even a smart router called "agents" which will use local models but if it thinks the request might require higher…

I built a router that lets you route between local and cloud models. Link in my profile.

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#32

Post by npn »

ddtaylor wrote: post #14

I enjoy the Qwen models a lot, but building things on top of them with OpenRouter has been painful. OpenRouter does a lot of great work and I really enjoy being able to use different models so easily. I like when a provider is phasing out an older model that still works for my needs and the price is much lower. It seems like such a good win-win. However, the problem is that many Qwen models have almost no capacity or…

I'm confused? Can you just define some presets and call them instead? With preset you can pinpoint a lot of things, especially the providers

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#33

Post by Zylokloto »

notnullorvoid wrote: post #27

It will be interesting to see the intersection of this with inference engines like FreeToken which improve distribution of work for MoE models across CPU/RAM and GPU/VRAM. If all it takes for a competitive model to run locally at good speeds is a used 3090 and some DDR4, then we might be in for the year of local AI. https://github.com/FlashML-org/FreeToken

You can already run it locally its just not the same.

It is still slow, a lot slower than what you are used to with claude and co.

And as soon as you increase context size, your memory requirements jump.

Then when it runs for 30 minutes for something claude needs 5, your device will get hot.

And even a used 3090 is apparently now between 1-2k.

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#34

Post by cpburns2009 »

hedora wrote: post #26

Time to dust off my 128GB strix halo (literally—it’s been dusty, and it’s running a bit warm these days). Any idea where this model sits according toquality benchmarks? Pre-bubble MSRP on this hardware was $1400, and it draws 200-ish watts, putting it down into consumer territory. I’m wondering if it can replace claude for llm-friendly coding tasks.

So back in the Qwen 3.5 release, the 122B-A10B model scored slightly better than the 27B model. I'd expect this new 125B-A6B to perform similarly to the recently released 27B. Qwen3.8 27B is supposed to rival Sonnet/Opus 4.6.

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#35

Post by Philpax »

cpburns2009 wrote: post #30
hasteg wrote: post #22

Earlier quoted context omitted.

As a 5090 owner and local model enthusiast, I was hoping it would be 35B A3B so I could run it myself =(.

You can run the 27B released last week. I haven't tried it yet myself but the 3.6 version runs great on my 5090.

Strongly recommend https://github.com/Neroued/ninfer, which can pull ~180 TPS on 5090 with 3.8, and 500 (!) with 3.6 35B-A3B.

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#36

Post by ljlolel »

ddtaylor wrote: post #14

I enjoy the Qwen models a lot, but building things on top of them with OpenRouter has been painful. OpenRouter does a lot of great work and I really enjoy being able to use different models so easily. I like when a provider is phasing out an older model that still works for my needs and the price is much lower. It seems like such a good win-win. However, the problem is that many Qwen models have almost no capacity or…

[dead]

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#38

Post by Casteil »

smcleod wrote: post #20
sscaryterry wrote: post #13

Earlier quoted context omitted.

I have a 128GB M5 Max, and it sucks at this stage. 50-70 tok/s might be something...

50-70tk/s is what I get on my m5 max on a 5-6bit Qwen 3.8 27B?

I don't know what black magic you're up to but I see more like 30-35t/s on a 16" M5 Max using 3.8:27b Q4, regardless of whether it's mlx or gguf.

qwen3.5:122b-a10b is significantly faster at around 60-65.

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#39

Post by Tuna-Fish »

hasteg wrote: post #22
cogman10 wrote: post #7

Wow. I wasn't expecting this. I thought they were going to do a 35B model instead.

As a 5090 owner and local model enthusiast, I was hoping it would be 35B A3B so I could run it myself =(.

The 27B one is great on a 5090.

This one is basically aimed at macs, Strix halo and DGX Spark.

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#40

Post by SwellJoe »

Finally, a reason to own a 128GB Strix Halo or GB10 device. Or a reason to consider the new Mac Studio.

I have a Strix Halo and dual 32GB GPUs in my desktop, and the latter is pretty much always better for running local models because it's quite a bit faster due to higher memory bandwidth. There simply haven't been any models that are better than Qwen 27B or Gemma 31B, which run comfortably in 64GB with big context.

And, MoE should make it run at a close to usable speed.

Post reply on HN