Qwen3-Next
qwen.ai
Qwen3-Next
1–10 of 240 posts
Re: Qwen3-Next
#2Seems impressive, i believe better architectures are really the path forward, i don't think you need more than 100B params taking this model and what GPT OSS 120B can acchieve
Re: Qwen3-Next
#3ERR_NAME_NOT_RESOLVED
Re: Qwen3-Next
#4Re: Qwen3-Next
#5Coolest part of Qwen3-Next, in my opinion, (after the linear attention parts) is that they do MTP without adding another un-embedding matrix.
Deepseek R1 also has a MTP layer (layer 61) https://huggingface.co/deepseek-ai/DeepSeek-R1/blob/main/mod...
But Deepseek R1 adds embed_tokens and shared_head.head tensors, which are [129280, 7168] or about 2GB in size at FP8.
Qwen3-Next doesn't have that: https://huggingface.co/Qwen/Qwen3-Next-80B-A3B-Instruct/blob...
So it saves a few GB in active parameters for MTP, which is a Big Deal. This is one of the changes that helps significantly speeds up inference.
Re: Qwen3-Next
#6Seems impressive, i believe better architectures are really the path forward, i don't think you need more than 100B params taking this model and what GPT OSS 120B can acchieve
New arch seems cool, and it's amazing that we have these published in the open.
That being said, qwen models are extremely overfit. They can do some things well, but they are very limited in generalisation, compared to closed models. I don't know if it's simply scale, or training recipes, or regimes. But if you test it ood the models utterly fail to deliver, where the closed models still provide value.
Re: Qwen3-Next
#7Complete newbie here - some questions, if I may!
This stuff can run on a local machine without internet access, correct?
And it can pretty much match Nano Banana? https://github.com/PicoTrex/Awesome-Nano-Banana-images/blob/...
Also -- what are the specs for a machine to run it (even if slowly!)
Re: Qwen3-Next
#8Seems impressive, i believe better architectures are really the path forward, i don't think you need more than 100B params taking this model and what GPT OSS 120B can acchieve
New arch seems cool, and it's amazing that we have these published in the open. That being said, qwen models are extremely overfit. They can do some things well, but they are very limited in generalisation, compared to closed models. I don't know if it's simply scale, or training recipes, or regimes. But if you test it ood the models utterly fail to deliver, where the closed models still provide value.
Could you give some practical examples? I don't know what Qwen's 36T-token training set is like, so I don't know what it's overfitting to...
Re: Qwen3-Next
#9Complete newbie here - some questions, if I may! This stuff can run on a local machine without internet access, correct? And it can pretty much match Nano Banana? https://github.com/PicoTrex/Awesome-Nano-Banana-images/blob/... Also -- what are the specs for a machine to run it (even if slowly!)
Isn't this one a text model
Re: Qwen3-Next
#10Complete newbie here - some questions, if I may! This stuff can run on a local machine without internet access, correct? And it can pretty much match Nano Banana? https://github.com/PicoTrex/Awesome-Nano-Banana-images/blob/... Also -- what are the specs for a machine to run it (even if slowly!)
Isn't this one a text model
Ah, maybe! I am lost reading this page with all the terminology