Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3: Think deeper, act faster

qwenlm.github.io

Post reply on HN
Print view View original

391–400 of 412 posts

Re: Qwen3: Think deeper, act faster

#391

Post by darepublic »

kaoD wrote: post #256
mrkeen wrote: post #253

Earlier quoted context omitted.

As they say, we shouldn't judge AI by the current state-of-the-art, but by how far and fast it's progressing. I can't wait to see future models get it even more wrong than that.

Personally (anecdata) I haven't experienced any practical progress in my day-to-day tasks for a long time, no matter how good they became at gaming the benchmarks. They keep being impressive at what they're good at (aggregating sources to solve a very well known problem) and terrible at what they're bad at (actually thinking through novel problems or old problems with few sources). E.g. all ChatGPT, Claude and Gemini…

Yes similar experience querying gpt about lesser known frameworks. Had o1 stone cold hallucinate some non existent methods I could find no trace of from googling. Would not budge on the matter either. Basically you have to provide the key insight yourself in these cases to get it unstuck, or just figure it out yourself. After its dug into a problem to some degree you get a feel for whether continued prompting on the subject is going to be helpful or just more churn

Re: Qwen3: Think deeper, act faster

#392

Post by a3w »

horhay wrote: post #96
kristofferR wrote: post #76

Earlier quoted context omitted.

You can't possibly use LLMs day to day if you think the benchmarks are solely gamed. Yes, there's been some cases, but the progress in real-life usage tracks the benchmarks overall. Gemini 2.5 Pro for example is absurdly more capable than models from a year ago.

They aren't lying in the way that LLMs have been seeing improvement, but benchmarks suggesting that LLMs are still scaling exponentially are not reflective of where they truly are.

AI 2027 had a good hint at what LLMs cannot do: robotics. So perhaps the singularity is near, after all, since this is pretty much my feeling too: LLMs are not skynet. But it is easier to pay people off in capitalism, than to engineer the torment nexus and threaten them into following. So it does not need killer robots+factories, if human have better chances in life by cooperating with LLMs instead.

Re: Qwen3: Think deeper, act faster

#393

Post by mks_shuffle »

Alifatisk wrote: post #332
mks_shuffle wrote: post #100

Does anyone have insights on the best approaches to compare reasoning models? It is often recommended to use a higher temperature for more creative answers and lower temperature values for more logical and deterministic outputs. However, I am not sure how applicable this advice is for reasoning models. For example, Deepseek-R1 and QwQ-32b recommend a temperature around 0.6, rather than lower values like 0.1–0.3. The…

Oh really? Should I adjust the temp to 0,6 on QwA-32B? Where did you get these numbers from?

These are recommendations provided on huggingface page under usage guidelines QwQ-32b: https://huggingface.co/Qwen/QwQ-32B DeepSeek-R1: https://huggingface.co/deepseek-ai/DeepSeek-R1

Re: Qwen3: Think deeper, act faster

#394

Post by CMay »

Havoc wrote: post #380
CMay wrote: post #378

The Qwen3 32B dense model just fails for me due to a template issue, but the Qwen3 30B A3B model does work. I think the more dense the model is at a small number of active parameters, the more sensitive a model can be to quantization. Only have 24GB of VRAM and have been using a 4-bit quantization which I use for most models. Qwen3 30B A3B is quick since it's MoE, but the results are just unreliable. It uses up all o…

>The Qwen3 32B dense model just fails for me due to a template issue, You're likely either using an old one or the broken on (repo second state). Try the ones unsloth uploaded Definitely works for me on LM studio

Ok, so Qwen3 32B works now with the update.

It seems much better than the Qwen3 30B A3B for quantized local use from what I can tell so far. Not sure yet how it compares to Gemma 3, but it's at least not clearly worse. It definitely does a much better job of formatting output in a friendlier way than Gemma does, but that's not as critical to me.

I suspect many people are getting even worse results out of the A3B than I did, since I saw downloads defaulting to 3-bit quants, but even at a higher quant, for local use it just isn't there yet.

I'm sure there are plenty of use cases for the low active parameter MoE models like sentiment analysis, summaries, etc, but for anything real I'll stick to the dense models. It makes me wonder if Qwen3 has similar problems that Llama 4 had, trying to be a big MoE model with low active parameters producing spotty results.

Qwen3 32B is quite usable, though. The problem I have with it so far is that it seems worse at instruction following, language translation and inferring the meaning of my prompt than Gemma 3. This isn't ideal, because if it can't follow instructions, you can't easily shape its reasoning/response to account for its issues.

One of my prompts simply asks it to do some translation and it occasionally feeds in Chinese characters. That's just not going to be usable for that scenario. Gemma 3's language consistency and quality is closer to production ready.

Gemma 3 does have its own problems with translation though, because if you instruct it to translate and what you want to translate is "what do you know?", it will instead go on talking about its capabilities rather than translating the language. You have to use a few tricks to prevent it from doing that.

Re: Qwen3: Think deeper, act faster

#395

Post by yencabulator »

furyofantares wrote: post #145
stavros wrote: post #98

I have a small physics-based problem I pose to LLMs. It's tricky for humans as well, and all LLMs I've tried (GPT o3, Claude 3.7, Gemini 2.5 Pro) fail to answer correctly. If I ask them to explain their answer, they do get it eventually, but none get it right the first time. Qwen3 with max thinking got it even more wrong than the rest, for what it's worth.

I similarly have a small, simple spatial reasoning problem that only reasoning models get right, and not all of them, and which Qwen3 on max reasoning still gets wrong. > I put a coin in a cup and slam it upside-down on a glass table. I can't see the coin because the cup is over it. I slide a mirror under the table and see heads. What will I see if I take the cup (and the mirror) away?

I think it's pretty random. qwen3:4b got it correct once, on re-run it told me the coin is actually behind the mirror, and then did this brilliant maneuver:

  - The question is **not** asking for the location of the coin, but its **identity**.
  - The coin is simply a **coin**, and the trick is in the riddle's wording.

  ---

  ### Final Answer:

  $$
  \boxed{coin}
  $$

Re: Qwen3: Think deeper, act faster

#396

Post by gparke »

bcoates wrote: post #389
markisus wrote: post #377

Earlier quoted context omitted.

You can stipulate conditions to make the solution work out in either direction. Suppose the bucket is the size of lake, and the leak is so miniscule that it takes many centuries to detect any loss. And also I need to keep the bucket full for a microsecond. In this case it is better to keep the bucket full, than to let it drain. Now suppose the bucket is made out of chain-link and any water you put into it immediately…

All heat that is lost must be replaced (we must input enough heat that the device returns to T_initial) Hotter objects lose heat faster, so the longer we delay restoring temperature (for a fixed resume time) the less heat is lost that will need replacement. Hotter objects require more energy to add another unit of heat, so the cooler we allow the device to get before re-heating (again, resume time is fixed) the more…

Does it depend on whether you know in advance _when_ you need it back at the hot temperature?

If you don’t think ahead and simply switch the heater back on when you need it, then you need the heater on for_longer_.

That means you have to pay back the energy you lost, but also the energy you lose during the reheating process. Maybe that’s the countervailing effect?

> Hotter objects require more energy to add another unit of heat

Not sure about this. A unit of heat is a unit energy, right? Maybe you were thinking of entropy?

Re: Qwen3: Think deeper, act faster

#397

Post by [deleted] »

bcoates wrote: post #389
markisus wrote: post #377

Earlier quoted context omitted.

You can stipulate conditions to make the solution work out in either direction. Suppose the bucket is the size of lake, and the leak is so miniscule that it takes many centuries to detect any loss. And also I need to keep the bucket full for a microsecond. In this case it is better to keep the bucket full, than to let it drain. Now suppose the bucket is made out of chain-link and any water you put into it immediately…

All heat that is lost must be replaced (we must input enough heat that the device returns to T_initial) Hotter objects lose heat faster, so the longer we delay restoring temperature (for a fixed resume time) the less heat is lost that will need replacement. Hotter objects require more energy to add another unit of heat, so the cooler we allow the device to get before re-heating (again, resume time is fixed) the more…

[deleted]

Re: Qwen3: Think deeper, act faster

#398

Post by RandyOrion »

aurareturn wrote: post #218
RandyOrion wrote: post #201

Earlier quoted context omitted.

YMMV. Parameter efficiency is an important consideration, if not the most important one, for local LLMs because of the hardware constraint. Do you guys really have GPUs with 80GB VRAM or M3 ultra with 512GB rams at home? If I can't run these ultra large MoEs locally, then these models mean nothing to me. I'm not a large LLM inference provider after all. What's more, you also lose the opportunities to fine-tune these…

What people actually want is something like GPT4o/o1 running locally. That's the dream for local LLM people. Running a 7b model for fun is not what people actually want. 7b models are very niche oriented.

About <10B LLMs, yes it's not that good. However, <10B is a range that allows many people to do their own tweaking and fine-tuning.

Re: Qwen3: Think deeper, act faster

#399

Post by hbbio »

Casteil wrote: post #335
ammo1662 wrote: post #273

Earlier quoted context omitted.

Just add "/no_think" in your prompt. https://qwenlm.github.io/blog/qwen3/#advanced-usages

FWIW, their readme states /nothink - and that's what works for me. >/think and /nothink instructions: Use those words in the system or user message to signify whether Qwen3 should think. In multi-turn conversations, the latest instruction is followed. https://github.com/QwenLM/Qwen3/blob/main/README.md

Thanks, /nothink works!

So, Qwen3 1.7b is about the same speed just slightly worse than Gemma3 4b which is pretty impressive.

Qwen3 4b passes all 200 tests and is much faster than Mistral Small 3.1 24b or Gemma3 27b.

Re: Qwen3: Think deeper, act faster

#400

Post by hbbio »

phh wrote: post #262
hbbio wrote: post #246

Earlier quoted context omitted.

I also have a benchmark that I'm using for my nanoagent[1] controllers. Qwen3 is impressive in some aspects but it thinks too much! Qwen3-0.6b is showing even better performance than Llama 3.2 3b... but it is 6x slower. The results are similar to Gemma3 4b, but the latter is 5x faster on Apple M3 hardware. So maybe, the utility is to run better models in cases where memory is the limiting factor, such as Nvidia GPUs?…

What's cool with those models is that you can tweak the thinking process, all the way down to "no thinking". It's maybe not available in your inference engine though

Now it is, thanks for suggesting. Qwen3 4b seems to be the best default model for usual steps.

https://github.com/hbbio/nanoagent/pull/1

Post reply on HN