looks like it's better than deepseek v4 flash
I agreed
Qwen3.8-Flash-Next
91–100 of 246 posts
Re: Qwen3.8-Flash-Next
#92Adding to my homelab stack, hopefully it doesn't overthink like the little model. Actually, hoping it thinks a bit less. Wait actually I'm really praying it reasons a bit more directly. But wait, I'm really sure that it must be a bit better.
You might already know this, but a large part of test-time compute / 'overthinking' is just letting the model do more passes, and refine its activation residuals more. For example, even if you make thinking tokens literally just '....' (absolutely meaningless; zero information), you still see significant performance improvements: https://arxiv.org/abs/2404.15758 and https://arxiv.org/abs/2607.22925 for some starters.…
Yeah I understand, it's my assumption that the actually/wait/but have a point. It doesn't reduce the fact that it increases the time for tasks substantially.
Re: Qwen3.8-Flash-Next
#93Adding to my homelab stack, hopefully it doesn't overthink like the little model. Actually, hoping it thinks a bit less. Wait actually I'm really praying it reasons a bit more directly. But wait, I'm really sure that it must be a bit better.
You’re absolutely right to be hopeful. Three honest possibilities, and I’ll be straight with you about each: 1. It overthinks — Just like the previous iteration. High confidence. 2. It doesn’t overthink — Improvement from the last model for your use case. Regression for others. 3. It sometimes overthinks — Best case all around. A feature, not an impairment. One final thing worth mentioning: (I made myself irrationall…
That's a caveat, and a real one.-
Re: Qwen3.8-Flash-Next
#94Earlier quoted context omitted.
I don't like these comparisons. Sure it is impressive, but it does not have a world knowledge of larger models. It has most of theirs intelligence.
In a lot of spaces, this is actually preferable. Ex - nodejs natively supports a huge set of typescript with built-in type stripping these days. But ask most hosted models to build a typescript project and they default to a heavy compile step, or a tool like tsx, ts-node, etc. Models with lots of "world knowledge" have a good chunk of that knowledge go stale, and there's no real way to refresh it without training a n…
> Personally, I'm really interested to see if we're headed towards a spot where the model is entirely distinct from the knowledge store.
This is what I've been trying to focus on with local AI for now. I've been trying to build all new documentation so it's more AI friendly. It's been pretty interesting. Qwen-35BA3B with a small prompt does a good job of surfacing what I'd consider institutional knowledge.
I've been trying to silo the docs I write from the model with a prompt that tells it not to use general knowledge unless asked to. From the anecdotal testing I did, Qwen-35BA3B is great for it. It does a really good job of following the prompt and calling tools, so I've been able to play around a lot to see what seems to work best.
Ultimately, I think one of the most effective uses of AI will be having a distinct knowledge store combined with an opinionated agent (and sub-agent) setup along with different models for each task.
Who owns the knowledge store is going to be the big caveat. Right now I think the big online models are trying for generic, persistent memory and I'd be very hesitant to let that happen. Think of having someone with a perfect memory following you around forever, but someone else has the ability to make them disappear. That's not a good situation.
Re: Qwen3.8-Flash-Next
#95Unsloth doesn't have all quant versions yet :-(
Re: Qwen3.8-Flash-Next
#96Does anyone have an idea how this might perform on a DGX Spark at longer contexts? I've been trying to investigate their performance with these medium-sized MoE models, but I'm seeing a lot of incomplete and conflicting information. The 273 GB/s bandwidth looks awfully bad on paper...
Given the new architecture, speeds are harder to estimate. Max would be ~40 tok/s.
Re: Qwen3.8-Flash-Next
#97Adding to my homelab stack, hopefully it doesn't overthink like the little model. Actually, hoping it thinks a bit less. Wait actually I'm really praying it reasons a bit more directly. But wait, I'm really sure that it must be a bit better.
You’re absolutely right to be hopeful. Three honest possibilities, and I’ll be straight with you about each: 1. It overthinks — Just like the previous iteration. High confidence. 2. It doesn’t overthink — Improvement from the last model for your use case. Regression for others. 3. It sometimes overthinks — Best case all around. A feature, not an impairment. One final thing worth mentioning: (I made myself irrationall…
> Three honest possibilities, and I’ll be straight with you about each
This. I don't know if the "honest answer" phrasing is part of the system prompt or alignment, but when people say "honestly" all the time I start wondering how honest they're being.
Re: Qwen3.8-Flash-Next
#98Adding to my homelab stack, hopefully it doesn't overthink like the little model. Actually, hoping it thinks a bit less. Wait actually I'm really praying it reasons a bit more directly. But wait, I'm really sure that it must be a bit better.
What about adding rtk proxy?
Re: Qwen3.8-Flash-Next
#99Earlier quoted context omitted.
You’re absolutely right to be hopeful. Three honest possibilities, and I’ll be straight with you about each: 1. It overthinks — Just like the previous iteration. High confidence. 2. It doesn’t overthink — Improvement from the last model for your use case. Regression for others. 3. It sometimes overthinks — Best case all around. A feature, not an impairment. One final thing worth mentioning: (I made myself irrationall…
That's a caveat, and a real one.-
But the reason why it remains load bearing is key.
Re: Qwen3.8-Flash-Next
#100It's in Unsloth Desktop already. Looks like it's 73GB, so 128GB Mac or Strix Halo etc will work. Exciting!
I only see a 1-bit quant posted on unsloth HF and it’s 72.5 GB. Is that what you mean? That’s much bigger than I expected. If you can’t run a 4 bit quant in on Strix Halo it becomes a lot less interesting. https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF
In their page they say it will need at least 112GB[0], so including context, that would be a tight fit. I'm also hoping I can make a q4 fit on my 128GB strix halo
[0]: https://unsloth.ai/docs/models/qwen3.8-next#qwen3.8-flash-ne...