Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3: Think deeper, act faster

qwenlm.github.io

Post reply on HN
Print view View original

301–310 of 412 posts

Re: Qwen3: Think deeper, act faster

#301

Post by artemisart »

furyofantares wrote: post #145
stavros wrote: post #98

I have a small physics-based problem I pose to LLMs. It's tricky for humans as well, and all LLMs I've tried (GPT o3, Claude 3.7, Gemini 2.5 Pro) fail to answer correctly. If I ask them to explain their answer, they do get it eventually, but none get it right the first time. Qwen3 with max thinking got it even more wrong than the rest, for what it's worth.

I similarly have a small, simple spatial reasoning problem that only reasoning models get right, and not all of them, and which Qwen3 on max reasoning still gets wrong. > I put a coin in a cup and slam it upside-down on a glass table. I can't see the coin because the cup is over it. I slide a mirror under the table and see heads. What will I see if I take the cup (and the mirror) away?

ChatGPT free gets it right without reasoning mode (still explained some steps) https://chatgpt.com/share/6810bc66-5e78-8001-b984-e4f71ee423...

Re: Qwen3: Think deeper, act faster

#302

Post by Filligree »

stavros wrote: post #261
oscord wrote: post #231

Earlier quoted context omitted.

Qwen3-32b did it pretty accurately it seems. Calculated heat loss over time going to ambient temp, offered to keep it at standby 100C for short breaks under 10 minutes. Shut down completely for longer breaks.

The correct answer is that it's always better to turn it off, though.

Unless you care about warmup time. LLMs have a habit of throwing in common-sense assumptions that you didn’t tell it to, so you have to be careful of that.

It’s not a bug. Outside of logic puzzles that’s a very good thing.

Re: Qwen3: Think deeper, act faster

#303

Post by mromanuk »

stavros wrote: post #98

I have a small physics-based problem I pose to LLMs. It's tricky for humans as well, and all LLMs I've tried (GPT o3, Claude 3.7, Gemini 2.5 Pro) fail to answer correctly. If I ask them to explain their answer, they do get it eventually, but none get it right the first time. Qwen3 with max thinking got it even more wrong than the rest, for what it's worth.

I was expecting a different outcome, that you tell us that Qwen3 nailed at first.

Re: Qwen3: Think deeper, act faster

#304

Post by stavros »

Filligree wrote: post #302
stavros wrote: post #261

Earlier quoted context omitted.

The correct answer is that it's always better to turn it off, though.

Unless you care about warmup time. LLMs have a habit of throwing in common-sense assumptions that you didn’t tell it to, so you have to be careful of that. It’s not a bug. Outside of logic puzzles that’s a very good thing.

No, warmup time doesn't change anything, I can simply factor it in.

It is a bug, because I asked it precisely what I wanted, and it gave the wrong answer. It didn't say anything about warmup time, it was just wrong.

Re: Qwen3: Think deeper, act faster

#305

Post by ljosifov »

I'm enjoying this ngl. :-) Alibaba_Qwen did themselves proud--top marks!

Qwen3-30B-A3B, a MoE 30B - with only 3B active at any one time I presume? - 4bit MLX in lmstudio, with speculative decoding via Qwen3-0.6B 8bit MLX, on an oldish M2 mbp first try delivered 24 tps(!!) -

24.29 tok/sec • 1953 tokens • 3.25s to first token • Stop reason: EOS Token Found • Accepted 1092/1953 draft tokens (55.9%)

Thank you to LMStudio, MLX and huggingface too. :-) After decades of not finding enough reasons for an MBP, suddenly ASI was it. And it's delivered beyond any expectations I had, already.

Did I mention I seem to have become NN PDP enthusiast, an AI maximalist? ;-) I thought them people over-excitable, if benevolent. Then the thought of trusting Trump-Putin on decisions like thermo-nuclear war ending us all, over ChatGPT and its reasoning offspring, converted me. AI is our only chance at existential salvation--ignore doom risk.

Re: Qwen3: Think deeper, act faster

#306

Post by anentropic »

simonw wrote: post #207

As is now traditional for new LLM releases, I used Qwen 3 (32B, run via Ollama on a Mac) to summarize this Hacker News conversation about itself - run at the point when it hit 112 comments. The results were kind of fascinating, because it appeared to confuse my system prompt telling it to summarize the conversation with the various questions asked in the post itself, which it tried to answer. I don't think it did a g…

This sounds like a task where you wouldn't want to use the 'thinking' mode

Re: Qwen3: Think deeper, act faster

#307

Post by croemer »

simonw wrote: post #220
littlestymaar wrote: post #213

Earlier quoted context omitted.

Aren't all Qwen models known to perform poorly with system prompt though?

I hadn't heard that, but it would certainly explain why the model made a mess of this task. Tried it again like this, using a regular prompt rather than a system prompt (with the https://github.com/simonw/llm-hacker-news plugin for the hn: prefix): llm -f hn:43825900 \ 'Summarize the themes of the opinions expressed here. For each theme, output a markdown header. Include direct "quotations" (with author attribution)…

Seems to truncate the input to only 2048 input tokens

Re: Qwen3: Think deeper, act faster

#309

Post by bboygravity »

apercu wrote: post #293
ggregoryarms wrote: post #288

Earlier quoted context omitted.

Building a basic static html landing page is ridiculously easy though. What js is even needed? If it's just an html file and maybe a stylesheet of course it's easy to host. You can apply 20 lines of css and have a decent looking page. These aren't hard problems.

> These aren't hard problems. So why do so many LLMs fail at them?

And humans also.

Re: Qwen3: Think deeper, act faster

#310

Post by laurent_du »

stavros wrote: post #98

I have a small physics-based problem I pose to LLMs. It's tricky for humans as well, and all LLMs I've tried (GPT o3, Claude 3.7, Gemini 2.5 Pro) fail to answer correctly. If I ask them to explain their answer, they do get it eventually, but none get it right the first time. Qwen3 with max thinking got it even more wrong than the rest, for what it's worth.

I do the same with a small math problem and so far only Qwen3 got it right (tested all thinking models). So your mileage may vary, as they say!
Post reply on HN