Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

modelscope.cn

Post reply on HN
Print view View original

161–170 of 178 posts

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#161

Post by pizza234 »

sosodev wrote: post #73
SwellJoe wrote: post #70

Earlier quoted context omitted.

I can do traditional server stuff on any old computer with a big hard disk and a decent amount of RAM. That's not worth $3500-$4000. When RAMpocalypse is over and we can buy a Strix Halo for under $2000 again, the math starts mathing. It becomes a pretty great desktop computer that also happens to run AI pretty well at a pretty good price.

Yeah, but that computer can’t also do the AI stuff. And not everybody has a desktop with multiple 32GB GPUs available. I’ll admit though I’m biased because I bought my board for $1600 back before the prices went crazy.

I spent around 5k on a server for "AI stuff" and it's currently doing no AI, because local LLMs (at least on systems with 32 GB VRAM) can only do only very basic stuff; this includes Qwen3.8 - in spite of the reverse engineering blog post, when I've tried Qwen to do a similar task, it flunked miserably. Additionally, I've read on some informal sources, the next step in quality is at 256 GB, not 128, which is very expensive (it's around 10k). 10k for privacy is... a toy for rich tinkerers, considering that most the people have their email on the cloud.

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#162

Post by pizza234 »

embedding-shape wrote: post #98
jubilanti wrote: post #94

Earlier quoted context omitted.

A 3060ti 8gb, released in 2020, has 448 GB/s of bandwidth compared to the Halo 256 GB/s The 3080ti is 912.4 GB/s

And the newly announced/launched Apple M6 has 170GB/s of unified memory bandwidth, meanwhile M5 Ultra gets 1.2TB/s of unified memory bandwidth. https://www.apple.com/newsroom/2026/08/apple-introduces-m6-a... Not sure if the first one is a typo on their press release, can't be just 170GB/s then be pushed for AI use, can it? Could be a different measurement I suppose...

Not a hardware engineer, but it's mainly because of RAM wires/channels (not implying that this is "simple" form an engineering perspective).

Using the published bandwidths, the math is 170 * 1 and 153 * 8.

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#163

Post by embedding-shape »

_ache_ wrote: post #120
embedding-shape wrote: post #118

Earlier quoted context omitted.

How long is a rope? Technically you could probably run it off a SSD, but it'll be slow as molasses. If you want it "fast", you want it all within GPU and VRAM, who knows what that'd be. If the engram parameters are separate, I guess it'd be like BF16 ~400 GB, FP8 ~200 GB, NVFP4 ~100GB. Otherwise maybe like ~300GB, ~150GB and ~70GB or alike, don't quote me that, only some guesses. The one who waits will see :)

I think a reasonable expectation of MAX requirement to claim "runable on consumer hardware" is to 32G VRAM and 128GB RAM and it run at +10tps.

Ok, so you already know what your expectations of the requirements are, and you aren't interested in more conservative perspectives, why do you ask to begin with?

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#164

Post by embedding-shape »

pizza234 wrote: post #162
embedding-shape wrote: post #98

Earlier quoted context omitted.

And the newly announced/launched Apple M6 has 170GB/s of unified memory bandwidth, meanwhile M5 Ultra gets 1.2TB/s of unified memory bandwidth. https://www.apple.com/newsroom/2026/08/apple-introduces-m6-a... Not sure if the first one is a typo on their press release, can't be just 170GB/s then be pushed for AI use, can it? Could be a different measurement I suppose...

Not a hardware engineer, but it's mainly because of RAM wires/channels (not implying that this is "simple" form an engineering perspective). Using the published bandwidths, the math is 170 * 1 and 153 * 8.

But 170GB/s is almost nothing? None of the RTX 50 series GPUs has that low bandwidth, you have to go back two generations of nvidia GPUs to get closer to that, and then it's the cheapest of the series, RTX 3050, which has ~170GB/s.

Even the GTX 1080, launched ten years ago, has double the bandwidth!

This must be some different way of measuring the bandwidth right? Since they explicitly say this for AI, but the numbers they share don't show that at all. Or I gravely misunderstand something here.

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#165

Post by jcsycombinator »

pizza234 wrote: post #161
sosodev wrote: post #73

Earlier quoted context omitted.

Yeah, but that computer can’t also do the AI stuff. And not everybody has a desktop with multiple 32GB GPUs available. I’ll admit though I’m biased because I bought my board for $1600 back before the prices went crazy.

I spent around 5k on a server for "AI stuff" and it's currently doing no AI, because local LLMs (at least on systems with 32 GB VRAM) can only do only very basic stuff; this includes Qwen3.8 - in spite of the reverse engineering blog post, when I've tried Qwen to do a similar task, it flunked miserably. Additionally, I've read on some informal sources, the next step in quality is at 256 GB, not 128, which is very exp…

If you can't get Qwen 3.8 27b to work well that's a skill issue.

You just suck at using ai

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#166

Post by embedding-shape »

pizza234 wrote: post #161
sosodev wrote: post #73

Earlier quoted context omitted.

Yeah, but that computer can’t also do the AI stuff. And not everybody has a desktop with multiple 32GB GPUs available. I’ll admit though I’m biased because I bought my board for $1600 back before the prices went crazy.

I spent around 5k on a server for "AI stuff" and it's currently doing no AI, because local LLMs (at least on systems with 32 GB VRAM) can only do only very basic stuff; this includes Qwen3.8 - in spite of the reverse engineering blog post, when I've tried Qwen to do a similar task, it flunked miserably. Additionally, I've read on some informal sources, the next step in quality is at 256 GB, not 128, which is very exp…

Same, I've had 96GB VRAM available for more than a year, and testing all the local LLMs as they become available, and read about people using models with Q2 quantization saying they get real work done, meanwhile I can't run 4x the size models with BF16 and get them to not eventually fuck up something.

Currently, I think most models are somewhat benchmaxxed, as my own benchmarks and others I know who also keep private benchmarks, don't show nearly as much needle movement as the press releases tend to want you to believe.

I can't wait for local LLMs to mature enough so I can move everything locally, but we're sadly not there yet today.

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#167

Post by jcsycombinator »

jubilanti wrote: post #152
SwellJoe wrote: post #108

Earlier quoted context omitted.

Enjoy Gemma 4 E2B at blistering speeds, I guess?

My point was more: this was the 2nd lowest end card from a generation 6 years ago, and it had way higher bandwidth than today's alleged flagship.

Small amounts of fast ram vs huge amounts of slower ram.

It costs more than the strix to just buy regular ddr5 ram sticks today.

Re: Qwen 3.8-Flash-Next releasing tomorrow (125B a6B)

#169

Post by embedding-shape »

jcsycombinator wrote: post #165
pizza234 wrote: post #161

Earlier quoted context omitted.

I spent around 5k on a server for "AI stuff" and it's currently doing no AI, because local LLMs (at least on systems with 32 GB VRAM) can only do only very basic stuff; this includes Qwen3.8 - in spite of the reverse engineering blog post, when I've tried Qwen to do a similar task, it flunked miserably. Additionally, I've read on some informal sources, the next step in quality is at 256 GB, not 128, which is very exp…

If you can't get Qwen 3.8 27b to work well that's a skill issue. You just suck at using ai

Which exact quantization you use of that particular model?

I've just spent two days testing it qualitatively (after the quantitative results were kind of bad, but promising), the official BF16 weights, and it keeps fucking up very basic things, like doing "rm -rf sessions/$id" without really checking or making sure $id is defined, so it ends up deleting more.

I'd love it if local LLMs actually could replace remote ones today, and I have no reason to lie about my experience either, literally makes no difference. But I too got hopefully reading the sentiment on the internet about Qwen 3.8, but it's just not there (yet) for local professional software development.

Verbatim logs from an open session where this literally just happened:

    Wait, let me reconsider. Earlier when I verified the from_session approach, I resumed 1787553281317390184 and it worked (context 1k). Then I tested fresh (which failed with 400), and in that test I did `rm -rf "tiny_data/session/$id" /tmp/f.txt` where $id was the fresh session's id (empty, since it failed). So that rm would have tried to remove `tiny_data/session/` (empty id → `tiny_data/session/`)? Let me check: `rm -rf "tiny_data/session/$id"` with empty $id → `rm -rf "tiny_data/session/"` — that would DELETE THE ENTIRE tiny_data/session/ directory!!!
    
    Oh no. That's what happened. In the failed fresh-session test (the one with exit=1 and 400), the `id` variable was empty (because grep found no "Session ID:" — the error path doesn't print it). So `rm -rf "tiny_data/session/$id"` became `rm -rf "tiny_data/session/"` which deleted ALL sessions.
Oh no indeed :)
Post reply on HN