Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

modelscope.cn

Post reply on HN
Print view View original

111–120 of 178 posts

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#111

Post by runeblaze »

ddtaylor wrote: post #14

I enjoy the Qwen models a lot, but building things on top of them with OpenRouter has been painful. OpenRouter does a lot of great work and I really enjoy being able to use different models so easily. I like when a provider is phasing out an older model that still works for my needs and the price is much lower. It seems like such a good win-win. However, the problem is that many Qwen models have almost no capacity or…

genuine question — how has fireworks or baseten or $reputable_inference_provider worked for your use cases? most production workload probably works fine with one of these and another set as fallback, at least so i think

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#113

Post by ekidd »

notnullorvoid wrote: post #51
Zylokloto wrote: post #33

Earlier quoted context omitted.

You can already run it locally its just not the same. It is still slow, a lot slower than what you are used to with claude and co. And as soon as you increase context size, your memory requirements jump. Then when it runs for 30 minutes for something claude needs 5, your device will get hot. And even a used 3090 is apparently now between 1-2k.

> It is still slow, a lot slower than what you are used to with claude and co. That really depends on the model, I run a few models locally. All at speeds comparable to or faster than Opus. In general we haven't reached the ceiling for what performance we can get out of consumer hardware. As evidence by FreeToken which hasn't even added MTP/speculative drafting support yet, which will add another boost. > Then when i…

> That really depends on the model, I run a few models locally. All at speeds comparable to or faster than Opus.

Yes, a lot of Qwen3.8 27B setups are actually quite snappy, as long as they fit 100% in VRAM. In my testing, I wouldn't go below 32GB of VRAM, though—you really want a 6-bit quant and 8-bit K/V quants minimum. I've seen too much weirdness out of 4-bit quants since Qwen3.8 shipped. I think it may be damaged more than 3.6 at similar levels of quantization?

If hyperscalers hadn't bought up almost all the fast RAM production for the next several years, 32GB of VRAM would be tolerably cheap—a lot by "home PC" standards, but not terrible by "professional tools" standards. Sadly, the RAM market is amazingly ugly right now.

> I doubt the timing differential here, but even still I run my 3090 pretty heavily with inference workloads and it stays cooler than when I use it for gaming.

Yeah, running inference on a laptop is likely to run quite hot. But in an ATX case with decent cooling, it's generally a lower load than gaming. One handy tip: Many Nvidia GPUs (and some from other manufacturers) support power limits. For example, limit a 5090 to 400W instead of 600W, and it will run much cooler. You might lose 11% off your tokens/sec (depending on the exact card).

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#114

Post by dackdel »

irthomasthomas wrote: post #17
ddtaylor wrote: post #14

I enjoy the Qwen models a lot, but building things on top of them with OpenRouter has been painful. OpenRouter does a lot of great work and I really enjoy being able to use different models so easily. I like when a provider is phasing out an older model that still works for my needs and the price is much lower. It seems like such a good win-win. However, the problem is that many Qwen models have almost no capacity or…

Openrouter was pretty great before prompt caching became common. Now it is extremely expensive for most individual workflows, unless you spend a lot of work customizing router preferences, and then you still get a worse cache hit rate than using the provider directly. I only keep $5-$10 in OR for occasional testing.

didnt stripe acquire open router? so i assumed its sunset.

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#115

Post by surcap526 »

SwellJoe wrote: post #70
sosodev wrote: post #65

Earlier quoted context omitted.

That’s only true if you think AI is the only reason to own a powerful and efficient server. Mine does plenty of traditional server stuff too.

I can do traditional server stuff on any old computer with a big hard disk and a decent amount of RAM. That's not worth $3500-$4000. When RAMpocalypse is over and we can buy a Strix Halo for under $2000 again, the math starts mathing. It becomes a pretty great desktop computer that also happens to run AI pretty well at a pretty good price.

[dead]

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#116

Post by kamranjon »

NitpickLawyer wrote: post #72
c16 wrote: post #63

+1 to the long list of people hoping for Qwen3.8-27b A3B.

They've said no moe for 3.8, and since they're already releasing a qwen4 early preview, they're probably focusing on that arch going forward.

Where did they say that? My understanding of this 3.8-Flash-Next release is that it's a MOE (as per the title of the posting here, 125B a6b)

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#117

Post by kamranjon »

Eric_WVGG wrote: post #60
pwython wrote: post #11

I was already rolling around the idea of a 128GB M5 Max MBP. Now this! A 4-bit MLX quant with 128k window should fit perfectly, in the 50-70 tok/s range.

Just out of curiosity, why run "local-local" when you could just set up a Mini or Studio at home and query it over http? [edit] whole conversation about this in another thread https://news.ycombinator.com/item?id=49433413 I’m personally considering retiring my MBP for a Studio + 15" Air whenever this MBP ages out.

I actually do this with my MBP - it's a LLM server when I'm working - and then when I'm not it's just a really great machine for video editing and other media work.

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#118

Post by embedding-shape »

_ache_ wrote: post #112

What will be the requirement, like 128G of RAM and 12G of VRAM ?

How long is a rope? Technically you could probably run it off a SSD, but it'll be slow as molasses. If you want it "fast", you want it all within GPU and VRAM, who knows what that'd be. If the engram parameters are separate, I guess it'd be like BF16 ~400 GB, FP8 ~200 GB, NVFP4 ~100GB. Otherwise maybe like ~300GB, ~150GB and ~70GB or alike, don't quote me that, only some guesses. The one who waits will see :)

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#119

Post by yjftsjthsd-h »

jubilanti wrote: post #94
SwellJoe wrote: post #40

Finally, a reason to own a 128GB Strix Halo or GB10 device. Or a reason to consider the new Mac Studio. I have a Strix Halo and dual 32GB GPUs in my desktop, and the latter is pretty much always better for running local models because it's quite a bit faster due to higher memory bandwidth. There simply haven't been any models that are better than Qwen 27B or Gemma 31B, which run comfortably in 64GB with big context.…

A 3060ti 8gb, released in 2020, has 448 GB/s of bandwidth compared to the Halo 256 GB/s The 3080ti is 912.4 GB/s

But it also has 8GB of RAM.

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#120

Post by _ache_ »

embedding-shape wrote: post #118
_ache_ wrote: post #112

What will be the requirement, like 128G of RAM and 12G of VRAM ?

How long is a rope? Technically you could probably run it off a SSD, but it'll be slow as molasses. If you want it "fast", you want it all within GPU and VRAM, who knows what that'd be. If the engram parameters are separate, I guess it'd be like BF16 ~400 GB, FP8 ~200 GB, NVFP4 ~100GB. Otherwise maybe like ~300GB, ~150GB and ~70GB or alike, don't quote me that, only some guesses. The one who waits will see :)

I think a reasonable expectation of MAX requirement to claim "runable on consumer hardware" is to 32G VRAM and 128GB RAM and it run at +10tps.
Post reply on HN