Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3-Next

qwen.ai

Post reply on HN
Print view View original

121–130 of 240 posts

Re: Qwen3-Next

#121

Post by amelius »

mmmllm wrote: post #96
amelius wrote: post #87

Earlier quoted context omitted.

If you can make an LLM solve a problem but from 100 different angles at the same time, that's worth something.

Isn't that essentially how the MoE models already work? Besides, if that were infinitely scalable, wouldn't we have a subset of super-smart models already at very high cost? Besides, this would only apply for very few use cases. For a lot of basic customer care work, programming, quick research, I would say LLMs are already quite good without running it 100X.

> if that were infinitely scalable, wouldn't we have a subset of super-smart models already at very high cost

The compute/intelligence curve is not a straight line. It's probably more a curve that saturates, at like 70% of human intelligence. More compute still means more intelligence. But you'll never reach 100% human intelligence. It saturates way below that.

Re: Qwen3-Next

#122

Post by mcrutcher »

mmmllm wrote: post #96
amelius wrote: post #87

Earlier quoted context omitted.

If you can make an LLM solve a problem but from 100 different angles at the same time, that's worth something.

Isn't that essentially how the MoE models already work? Besides, if that were infinitely scalable, wouldn't we have a subset of super-smart models already at very high cost? Besides, this would only apply for very few use cases. For a lot of basic customer care work, programming, quick research, I would say LLMs are already quite good without running it 100X.

MoE models are pretty poorly named since all the "experts" are "the same". They're probably better described as "sparse activation" models. MoE implies some sort of "heterogenous experts" that a "thalamus router" is trained to use, but that's not how they work.

Re: Qwen3-Next

#123

Post by mmmllm »

idopmstuff wrote: post #101
mmmllm wrote: post #82

Earlier quoted context omitted.

Sure but where is the demand going to come from? LLMs are already in every google search, in Whatsapp/Messenger, throughout Google workspace, Notion, Slack, etc. ChatGPT already has a billion users. Plus penetration is already very high in the areas where they are objectively useful: programming, customer care etc. I just don't see where the 100-1000x demand comes from to offset this. Would be happy to hear other vie…

> Plus penetration is already very high in the areas where they are objectively useful: programming, customer care etc. Is that true? BLS estimates of customer service reps in the US is 2.8M ( https://www.bls.gov/oes/2023/may/oes434051.htm ), and while I'll grant that's from 2023, I would wager a lot that the number is still above 2M. Similarly, the overwhelming majority of software developers haven't lost their jobs…

Fair point - although there are already so many customer facing chatbots using LLMs rolled out already. Zendesk, Intercom, Hubspot, Salesforce service cloud all have AI features built into their workflows. I wouldn't say penetration is near the peak but it's also not early stage at this point.

In any case, AI is not capable of fully replacing customer care. It will make it more efficient but the non-deterministic nature of LLMs mean that they need to be supervised for complex cases.

Besides, I still think even the inference demand for customer care or programming will be small in the grand scheme of things. EVERY Google search (and probably every gmail email) is already passed through an LLM - the demand for that alone is immense.

I'm not saying demand won't increase, I just don't see how demand increases so much that it offsets the efficiency gains to such an extent that Oracle etc are planning tens or hundreds of times the need for compute in the next couple of years. Or at least I am skeptical of it to say the least.

Re: Qwen3-Next

#124

Post by mmmllm »

mirekrusin wrote: post #100
mmmllm wrote: post #96

Earlier quoted context omitted.

Isn't that essentially how the MoE models already work? Besides, if that were infinitely scalable, wouldn't we have a subset of super-smart models already at very high cost? Besides, this would only apply for very few use cases. For a lot of basic customer care work, programming, quick research, I would say LLMs are already quite good without running it 100X.

MoE is something different - it's a technique to activate just a small subset of parameters during inference. Whatever is good enough now, can be much better for the same cost (time, computation, actual cost). People will always choose better over worse.

Thanks, I wasn't aware of that. Still - why isn't there a super expensive OpenAI model that uses 1,000 experts and comes up with way better answers? Technically that would be possible to build today. I imagine it just doesn't deliver dramatically better results.

Re: Qwen3-Next

#125

Post by eldenring »

moffkalast wrote: post #42
jychang wrote: post #25

Earlier quoted context omitted.

Speculative decoding! It makes inference a LOT faster. Instead of generating tokens one at a time, you generate the second one as well, and then use speculative decoding on that second token (instead of having it be produced by a draft model like Qwen 0.6b). If the token is checked and is correct, then the 2nd token gets generated MUCH faster. If it's wrong, you have to generate it again the normal way (a lot slower…

Hmm but isn't the checking only required because the draft model is not the same model and can only speculate what the main one is thinking, hence the name? If the main model generates two tokens itself, then how can it be wrong about its own predictions?

the 2nd token is generated without knowing what token was chosen for the 1st token

Re: Qwen3-Next

#126

Post by WhitneyLand »

irthomasthomas wrote: post #31
dev_hugepages wrote: post #23

Earlier quoted context omitted.

memorized: https://www.asciiart.eu/cartoons/spongebob-squarepants

Naturally. That's how LLMs work. During training you measure the loss, the difference between the model output and the ground-truth and try to minimize it. We prize models for their ability to learn. Here we can see that the large model does a great job at learning to draw bob, while the small model performs poorly.

Not really.

Typically less than 1% of training data is memorized.

Re: Qwen3-Next

#127

Post by mmmllm »

stingraycharles wrote: post #109
mmmllm wrote: post #66

The same week Oracle is forecasting huge data center demand and the stock is rallying. If these 10x gains in efficiency hold true then this could lead to a lot less demand for Nvidia, Oracle, Coreweave etc

Absolutely not; the trends have proven that people will just pay for the best quality they can get, and keep paying roughly the same money. Every time a new model is released, people abandon the old, lower quality model (even when it’s priced less), and instead prefer to pay the same for a better model. The same will happen with this.

Sure but the money people are paying right now isn't that much in the grand scheme of things. OpenAI is expecting 13bn in revenue this year. AWS made over 100bn last year. So unless they pay a lot more, or they find customers outside of programmers, designers, etc who are willing to pay for the best quality, I don't see how it grows as fast as it needs to (I'm not saying it won't increase, just not at the rate expected by the data center providers)

Re: Qwen3-Next

#129

Post by sailingparrot »

mmmllm wrote: post #66

The same week Oracle is forecasting huge data center demand and the stock is rallying. If these 10x gains in efficiency hold true then this could lead to a lot less demand for Nvidia, Oracle, Coreweave etc

People said the same thing for deepseek-r1, and nothing changed.

If you come up with a way to make the current generation of models 10x more efficient, then everyone just moves to train a 10x bigger model. There isn’t a size of model where the players are going to be satisfied at and not go 10x bigger. Not as long as scaling still pays off (and it does today).

Re: Qwen3-Next

#130

Post by taminka »

vessenes wrote: post #118
mmmllm wrote: post #82

Earlier quoted context omitted.

Sure but where is the demand going to come from? LLMs are already in every google search, in Whatsapp/Messenger, throughout Google workspace, Notion, Slack, etc. ChatGPT already has a billion users. Plus penetration is already very high in the areas where they are objectively useful: programming, customer care etc. I just don't see where the 100-1000x demand comes from to offset this. Would be happy to hear other vie…

We are nearly infinitely far away from saturating compute demand for inference. Case in point; I'd like something that realtime assesses all the sensors and API endpoints of stuff in my home and as needed bubbles up summaries, diaries, and emergency alerts. Right now that's probably a single H200, and well out of my "value range". The number of people in the world that do this now at scale is almost certainly less th…

absolutely nobody wants or needs a fucking thermostat diary lmao, and the few ppl that do will have zero noticeable impact on world's compute demands, i'm begging ppl in on hn to touch grass or speak to an average person every now and then lol
Post reply on HN