Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3: Think deeper, act faster

qwenlm.github.io

Post reply on HN
Print view View original

201–210 of 412 posts

Re: Qwen3: Think deeper, act faster

#201

Post by RandyOrion »

aurareturn wrote: post #188
RandyOrion wrote: post #137

For ultra large MoEs from deepseek and llama 4, fine-tuning on these models is becoming increasingly impossible for hobbyists and local LLM users. Small and dense models are what local people really need. Although benchmaxxing is not good, I still find this release valuable. Thank you Qwen.

Small and dense models are what local people really need. Disagreed. Small and dense is dumber and slower for local inferencing. MoEs is what people actually want on local.

YMMV.

Parameter efficiency is an important consideration, if not the most important one, for local LLMs because of the hardware constraint.

Do you guys really have GPUs with 80GB VRAM or M3 ultra with 512GB rams at home? If I can't run these ultra large MoEs locally, then these models mean nothing to me. I'm not a large LLM inference provider after all.

What's more, you also lose the opportunities to fine-tune these MoEs when it's already hard to do inference with these MoEs.

Re: Qwen3: Think deeper, act faster

#203

Post by paradite »

Thinking takes way too long for it to be useful in practice.

It takes 5 minutes to generate first non-thinking token in my testing for a slightly complex task via Parasail and Deepinfra on OpenRouter.

https://x.com/paradite_/status/1917067106564379070

Update:

Finally got it work after waiting for 10 minutes.

Published my eval result, surprisingly non-thinking version did slightly better on visualization task: https://x.com/paradite_/status/1917087894071873698

Re: Qwen3: Think deeper, act faster

#204

Post by baxtr »

stavros wrote: post #98

I have a small physics-based problem I pose to LLMs. It's tricky for humans as well, and all LLMs I've tried (GPT o3, Claude 3.7, Gemini 2.5 Pro) fail to answer correctly. If I ask them to explain their answer, they do get it eventually, but none get it right the first time. Qwen3 with max thinking got it even more wrong than the rest, for what it's worth.

This reads like a great story with a tragic ending!

Re: Qwen3: Think deeper, act faster

#205

Post by CamperBob2 »

whimsicalism wrote: post #122
refulgentis wrote: post #61

Earlier quoted context omitted.

4 bit is absolutely fine . I know this is crazy to here because the big iron folks still debate 16 vs 32 and 8 vs 16 is near verboten in public conversation. I contribute to llama.cpp and have seen many many efforts to measure evaluation perf of various quants, and no matter which way it was sliced (ranging from subjective volunteers doing A/B voting on responses over months, to objective object perplexity loss) Q4 i…

> 8 vs 16 is near verboten in public conversation. i mean, deepseek is fp8

Not only that, but the 1.58 bit Unsloth dynamic quant is uncannily powerful.

Re: Qwen3: Think deeper, act faster

#206

Post by vintermann »

jean- wrote: post #152
omneity wrote: post #106

Excellent release by the Qwen team as always. Pretty much the best open-weights model line so far. In my early tests however, several of the advertised languages are not really well supported and the model is outputting something that only barely resembles them. Probably a dataset quality issue for low-resource languages that they cannot personally check for, despite the “119 languages and dialects” claim.

Indeed, I tried several low-resource Romance languages they claim to support and performance is abysmal.

What size/quantification level? IME, small language performance is one of the things that really suffers from the various tricks that are used to reduce size.

Re: Qwen3: Think deeper, act faster

#207

Post by simonw »

As is now traditional for new LLM releases, I used Qwen 3 (32B, run via Ollama on a Mac) to summarize this Hacker News conversation about itself - run at the point when it hit 112 comments.

The results were kind of fascinating, because it appeared to confuse my system prompt telling it to summarize the conversation with the various questions asked in the post itself, which it tried to answer.

I don't think it did a great job of the task, but it's still interesting to see its "thinking" process here: https://gist.github.com/simonw/313cec720dc4690b1520e5be3c944...

Re: Qwen3: Think deeper, act faster

#208

Post by Lucasoato »

furyofantares wrote: post #145
stavros wrote: post #98

I have a small physics-based problem I pose to LLMs. It's tricky for humans as well, and all LLMs I've tried (GPT o3, Claude 3.7, Gemini 2.5 Pro) fail to answer correctly. If I ask them to explain their answer, they do get it eventually, but none get it right the first time. Qwen3 with max thinking got it even more wrong than the rest, for what it's worth.

I similarly have a small, simple spatial reasoning problem that only reasoning models get right, and not all of them, and which Qwen3 on max reasoning still gets wrong. > I put a coin in a cup and slam it upside-down on a glass table. I can't see the coin because the cup is over it. I slide a mirror under the table and see heads. What will I see if I take the cup (and the mirror) away?

I tried with the thinking option on and it gets into some networking errors, if you don’t turn on the thinking it guesses the answer correctly.

> Summary:

- Mirror shows: *Heads* → That's the *bottom face* of the coin. - So actual top face (visible when cup is removed): *Tails*

Final answer: *You will see tails.*

Re: Qwen3: Think deeper, act faster

#209

Post by jasonjmcghee »

I've been testing the unsloth quantization: Qwen3-235B-A22B-Q2_K_L

It is by far the best local model I've ever used. Very impressed so far.

Llama 4 was a massive disappointment, so I'm having a blast.

Claude Sonnet 3.7 is still better though.

---

Also very impressed with qwen3-30b-a3b - so fast for how smart it is (i am using the 0.6b for speculative decoding). very fun to use.

---

I'm finding that the models want to give over-simplified solutions, and I was initially disappointed, but I added some stuff about how technical solutions should be written in the system prompt and they are being faithful to it.

Re: Qwen3: Think deeper, act faster

#210

Post by hmottestad »

furyofantares wrote: post #145
stavros wrote: post #98

I have a small physics-based problem I pose to LLMs. It's tricky for humans as well, and all LLMs I've tried (GPT o3, Claude 3.7, Gemini 2.5 Pro) fail to answer correctly. If I ask them to explain their answer, they do get it eventually, but none get it right the first time. Qwen3 with max thinking got it even more wrong than the rest, for what it's worth.

I similarly have a small, simple spatial reasoning problem that only reasoning models get right, and not all of them, and which Qwen3 on max reasoning still gets wrong. > I put a coin in a cup and slam it upside-down on a glass table. I can't see the coin because the cup is over it. I slide a mirror under the table and see heads. What will I see if I take the cup (and the mirror) away?

Tried it with o1-pro:

> You’ll find that the actual face of the coin under the cup is tails. Seeing “heads” in the mirror from underneath indicates that, on top, the coin is really tails‑up.

Post reply on HN