Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3-Next

qwen.ai

Post reply on HN
Print view View original

101–110 of 240 posts

Re: Qwen3-Next

#101

Post by idopmstuff »

mmmllm wrote: post #82
amelius wrote: post #70

Earlier quoted context omitted.

https://en.wikipedia.org/wiki/Jevons_paradox

Sure but where is the demand going to come from? LLMs are already in every google search, in Whatsapp/Messenger, throughout Google workspace, Notion, Slack, etc. ChatGPT already has a billion users. Plus penetration is already very high in the areas where they are objectively useful: programming, customer care etc. I just don't see where the 100-1000x demand comes from to offset this. Would be happy to hear other vie…

> Plus penetration is already very high in the areas where they are objectively useful: programming, customer care etc.

Is that true? BLS estimates of customer service reps in the US is 2.8M (https://www.bls.gov/oes/2023/may/oes434051.htm), and while I'll grant that's from 2023, I would wager a lot that the number is still above 2M. Similarly, the overwhelming majority of software developers haven't lost their jobs to AI.

A sufficiently advanced LLM will be able to replace most, if not all of those people. Penetration into those areas is very low right now relative to where it could be.

Re: Qwen3-Next

#102

Post by Alifatisk »

pilotneko wrote: post #92
Alifatisk wrote: post #65

Alibaba keeps releasing gold content I just tried Qwen3-Next-80B-A3B on Qwen chat, and it's fast! The quality seem to match Qwen3-235B-A22B. Quite impressive how they achieved this. Can't wait for the benchmarks at Artificial analysis According to Qwen Chat, Qwen3-Next has the following limits: Maximum context length: 262,144 tokens Max summary generation length: 32,768 tokens This is 2x higher on context length and…

If you read the model card, Qwen3-Next can be extended to 1M context length with YaRN. > Qwen3-Next natively supports context lengths of up to 262,144 tokens. For conversations where the total length (including both input and output) significantly exceeds this limit, we recommend using RoPE scaling techniques to handle long texts effectively. We have validated the model's performance on context lengths of up to 1 mil…

> If you read the model card, Qwen3-Next can be extended to 1M context length with YaRN.

I read the article, but as I said Qwen chat only provides up to 262k tokens in context length, so I'll stick with Qwen2.5 Turbo which supports 1M tokens.

I am not in a position where I can self-host yet

Re: Qwen3-Next

#103

Post by mdp2021 »

mmmllm wrote: post #66

The same week Oracle is forecasting huge data center demand and the stock is rallying. If these 10x gains in efficiency hold true then this could lead to a lot less demand for Nvidia, Oracle, Coreweave etc

The real quality demand needs is not there, so more processing is very probably needed, so efficiency gains may allow the extra processing.

(A string example read today of Real quality demand needs: the administration of Albania wants some sort of automated Cabinet Minister. Not just an impartial and incorruptible algorithm (what we normally try to do with deterministic computation): a "minister". Good luck with that.)

Re: Qwen3-Next

#104

Post by mdp2021 »

irthomasthomas wrote: post #31
dev_hugepages wrote: post #23

Earlier quoted context omitted.

memorized: https://www.asciiart.eu/cartoons/spongebob-squarepants

Naturally. That's how LLMs work. During training you measure the loss, the difference between the model output and the ground-truth and try to minimize it. We prize models for their ability to learn. Here we can see that the large model does a great job at learning to draw bob, while the small model performs poorly.

> That's how LLMs work

And that is also exactly how we want them not to work: we want them to be able to solve new problems. (Because Pandora's box is open, and they are not sold as a flexible query machine.)

"Where was Napoleon born": easy. "How to resolve the conflict effectively": hard. Solved problems are interesting to students. Professionals have to deal with non trivial ones.

Re: Qwen3-Next

#105

Post by davidweatherall »

Alifatisk wrote: post #65

Alibaba keeps releasing gold content I just tried Qwen3-Next-80B-A3B on Qwen chat, and it's fast! The quality seem to match Qwen3-235B-A22B. Quite impressive how they achieved this. Can't wait for the benchmarks at Artificial analysis According to Qwen Chat, Qwen3-Next has the following limits: Maximum context length: 262,144 tokens Max summary generation length: 32,768 tokens This is 2x higher on context length and…

Their proprietary models are very good too and go under the radar, they never seem to appear on any benchmarks. Qwen3-coder-plus is significantly better than their open source qwen3, Qwen3 max also rivals the SOTA models

Re: Qwen3-Next

#106

Post by binary132 »

I’ve been using gpt-oss-120B with CPU MoE offloading on a 24GB GPU and it’s very usable. Excited to see if I can get good results on this now!

Re: Qwen3-Next

#107

Post by ACCount37 »

mmmllm wrote: post #66

The same week Oracle is forecasting huge data center demand and the stock is rallying. If these 10x gains in efficiency hold true then this could lead to a lot less demand for Nvidia, Oracle, Coreweave etc

No. The gains in inference and training efficiency are going to be absorbed by frontier LLM labs being more willing to push more demanding and capable models to the end users, increase reasoning token budgets, etc.

Re: Qwen3-Next

#108

Post by stingraycharles »

jychang wrote: post #25
puilp0502 wrote: post #17

Earlier quoted context omitted.

What kind of benefit does Multi-Token Prediction bring to the inference side? Is it only relevant in pretraining efficiency?

Speculative decoding! It makes inference a LOT faster. Instead of generating tokens one at a time, you generate the second one as well, and then use speculative decoding on that second token (instead of having it be produced by a draft model like Qwen 0.6b). If the token is checked and is correct, then the 2nd token gets generated MUCH faster. If it's wrong, you have to generate it again the normal way (a lot slower…

Because then the second token only needs to be checked, not generated, as it’s already generated? And it’s much faster to generate multiple tokens at the same time than one at a time? Is that the idea?

I’m not an expert on LLMs, just a user.

Re: Qwen3-Next

#109

Post by stingraycharles »

mmmllm wrote: post #66

The same week Oracle is forecasting huge data center demand and the stock is rallying. If these 10x gains in efficiency hold true then this could lead to a lot less demand for Nvidia, Oracle, Coreweave etc

Absolutely not; the trends have proven that people will just pay for the best quality they can get, and keep paying roughly the same money.

Every time a new model is released, people abandon the old, lower quality model (even when it’s priced less), and instead prefer to pay the same for a better model.

The same will happen with this.

Re: Qwen3-Next

#110

Post by lcnPylGDnU4H9OF »

Razengan wrote: post #80
jychang wrote: post #5

Coolest part of Qwen3-Next, in my opinion, (after the linear attention parts) is that they do MTP without adding another un-embedding matrix. Deepseek R1 also has a MTP layer (layer 61) https://huggingface.co/deepseek-ai/DeepSeek-R1/blob/main/mod... But Deepseek R1 adds embed_tokens and shared_head.head tensors, which are [129280, 7168] or about 2GB in size at FP8. Qwen3-Next doesn't have that: https://huggingface.co…

Could someone kindly point to a convenient all-on-one ELI5 of all these words? :')

The best primer I've seen is Andrej Karpathy's first video in his "zero to hero" series. It's worth following along with your own practice.

https://karpathy.ai/zero-to-hero.html

Post reply on HN