Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3-Next

qwen.ai

Post reply on HN
Print view View original

151–160 of 240 posts

Re: Qwen3-Next

#151

Post by patates »

kridsdale3 wrote: post #85
gizmodo59 wrote: post #71

Earlier quoted context omitted.

My take on long context for many frontier models is not about support but the accuracy drops drastically as you increase the context. Even if a model claims to support 10M context, reality is it doesn’t perform well when you saturate. Curious to hear others perspective on this

This is my experience with Gemini. Yes, I really can put an entire codebase and all the docs and pre-dev discussions and all the inter-engineer chat logs in there. I still see the model becoming more intoxicated as turn count gets high.

I use repomix to pack a full repository as an xml file and it works wonders. System prompt is very simple:

please don't add any comments in the code unless explicitly asked to, including the ones that state what you changed. do not modify/remove any existing comments as long as they are valid. also output the full files that are changed (not the untouched ones), and no placeholders like "no change here" etc. do not output the xml parts in the output.xml file. focus on the individual files. before and after outputting code, write which file it would be and the path (not as a comment in the code but instead, before and after outputting code).

Attached is a 400k token xml file, being the output of:

https://pastebin.com/raw/SH6JHteg

Main prompt is a general description of the feature needed and PDF exports from figma.

All done for free in aistudio and I consistently get better results than the people using claude code.

Re: Qwen3-Next

#152

Post by [deleted] »

lcnPylGDnU4H9OF wrote: post #120
seunosewa wrote: post #73

Earlier quoted context omitted.

He is definitely talking about Llama4.

> last year > dense > 405B model Llama4 does not match any of these details. Maybe the commenter thinks their comment is about Llama4 (I don't see a reason to believe so) but readers familiar with these details know they are referring to Llama3.1.

[deleted]

Re: Qwen3-Next

#153

Post by ehsanu1 »

davidweatherall wrote: post #105
Alifatisk wrote: post #65

Alibaba keeps releasing gold content I just tried Qwen3-Next-80B-A3B on Qwen chat, and it's fast! The quality seem to match Qwen3-235B-A22B. Quite impressive how they achieved this. Can't wait for the benchmarks at Artificial analysis According to Qwen Chat, Qwen3-Next has the following limits: Maximum context length: 262,144 tokens Max summary generation length: 32,768 tokens This is 2x higher on context length and…

Their proprietary models are very good too and go under the radar, they never seem to appear on any benchmarks. Qwen3-coder-plus is significantly better than their open source qwen3, Qwen3 max also rivals the SOTA models

Are these actually different models vs just different names from the open weights releases?

Re: Qwen3-Next

#154

Post by mmmllm »

vessenes wrote: post #118
mmmllm wrote: post #82

Earlier quoted context omitted.

Sure but where is the demand going to come from? LLMs are already in every google search, in Whatsapp/Messenger, throughout Google workspace, Notion, Slack, etc. ChatGPT already has a billion users. Plus penetration is already very high in the areas where they are objectively useful: programming, customer care etc. I just don't see where the 100-1000x demand comes from to offset this. Would be happy to hear other vie…

We are nearly infinitely far away from saturating compute demand for inference. Case in point; I'd like something that realtime assesses all the sensors and API endpoints of stuff in my home and as needed bubbles up summaries, diaries, and emergency alerts. Right now that's probably a single H200, and well out of my "value range". The number of people in the world that do this now at scale is almost certainly less th…

Sure but if that inference cost went to 1%, then Oracle and Nvidia's business model would be bust. So you agree with me?

Re: Qwen3-Next

#155

Post by Leynos »

mmmllm wrote: post #124
mirekrusin wrote: post #100

Earlier quoted context omitted.

MoE is something different - it's a technique to activate just a small subset of parameters during inference. Whatever is good enough now, can be much better for the same cost (time, computation, actual cost). People will always choose better over worse.

Thanks, I wasn't aware of that. Still - why isn't there a super expensive OpenAI model that uses 1,000 experts and comes up with way better answers? Technically that would be possible to build today. I imagine it just doesn't deliver dramatically better results.

That's what GPT-5 Pro and Grok 4 Heavy do. Those are the ones you pay triple digit USD a month for.

Re: Qwen3-Next

#156

Post by jug »

> The Qwen3-Next-80B-A3B-Instruct performs comparably to our flagship model Qwen3-235B-A22B-Instruct-2507

I'm skeptical about these claims. How can this be? Wouldn't there be massive loss of world knowledge? I'm particularly skeptical because a recent trend in Q2 2025 has been benchmaxxing.

Re: Qwen3-Next

#157

Post by dragonwriter »

jug wrote: post #156

> The Qwen3-Next-80B-A3B-Instruct performs comparably to our flagship model Qwen3-235B-A22B-Instruct-2507 I'm skeptical about these claims. How can this be? Wouldn't there be massive loss of world knowledge? I'm particularly skeptical because a recent trend in Q2 2025 has been benchmaxxing.

> I'm skeptical about these claims. How can this be?

More efficient architecture.

> Wouldn't there be massive loss of world knowledge?

If you assume equally efficient architecture and no other salient differences, yes, that’s what you’d expect from a smaller model.

Re: Qwen3-Next

#158

Post by mtone »

vessenes wrote: post #118
mmmllm wrote: post #82

Earlier quoted context omitted.

Sure but where is the demand going to come from? LLMs are already in every google search, in Whatsapp/Messenger, throughout Google workspace, Notion, Slack, etc. ChatGPT already has a billion users. Plus penetration is already very high in the areas where they are objectively useful: programming, customer care etc. I just don't see where the 100-1000x demand comes from to offset this. Would be happy to hear other vie…

We are nearly infinitely far away from saturating compute demand for inference. Case in point; I'd like something that realtime assesses all the sensors and API endpoints of stuff in my home and as needed bubbles up summaries, diaries, and emergency alerts. Right now that's probably a single H200, and well out of my "value range". The number of people in the world that do this now at scale is almost certainly less th…

Do you really need a H200 for this? Seems like something a consumer GPU could do. Smaller models might be ideal [0] as they don't require extensive world knowledge and are much more cost efficient/faster.

Why can't you build this today?

[0]: https://arxiv.org/pdf/2506.02153 Small Language Models are the Future of Agentic AI (Nvidia)

Re: Qwen3-Next

#159

Post by littlestymaar »

vessenes wrote: post #114
Razengan wrote: post #80

Earlier quoted context omitted.

Could someone kindly point to a convenient all-on-one ELI5 of all these words? :')

Unfortunately, no. The industry is moving super quickly, and spinning up new ideas on the backs of old ones at a fast rate. If you want to understand what's going on, I think the best thing to do is some intro courses, train and design some smaller models directly, get a list of core papers and concepts from Claude/Chat/Gemini, and then as you read something like this, if you don't know the acronym (In this case: MTP…

> Unfortunately, no. The industry is moving super quickly, and spinning up new ideas on the backs of old ones at a fast rate.

I don't think it move this fast.

I mean there is very little fundamental differences between GPT-2 and gpt-oss-120b, it's just about incremental improvement that don't change much to the full picture (using a variation of the attention architecture and masking, a different activation function, the positional encoding and changing the NLP layers to a sparse “mixture of expert”), at the end of the day, from Mistral to Deepseek going through llama and Qwen3 it's always the same stack of transformers layers with slight variations between two architectures.

This Qwen3-Next is special though, as it's the first time a major player is releasing something that different (lesser players have made hybrid architecture LLMs for the past two years, but when it comes to language models, IBM really isn't comparable to Alibaba). This is what I expected Llama4 to be.

Re: Qwen3-Next

#160

Post by siliconc0w »

All these new datacenters are going to be a huge sunk cost. Why would you pay OpenAI when you can host your own hyper efficient Chinese model for like 90% less cost at 90% of the performance. At that is compared to today's subsidized pricing, which they can't keep up forever.
Post reply on HN