Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3: Think deeper, act faster

qwenlm.github.io

Post reply on HN
Print view View original

181–190 of 412 posts

Re: Qwen3: Think deeper, act faster

#181

Post by daemonologist »

bionhoward wrote: post #139
daemonologist wrote: post #32

Earlier quoted context omitted.

The ultra-simplified napkin math is 1 GB (V)RAM per 1 billion parameters, at a 4-5 bit-per-weight quantization. This usually gives most of the performance of the full size model and leaves a little bit of room for context, although not necessarily the full supported size.

Wouldn’t it be 1GB (billion bytes) per billion parameters when each parameter is 1 byte (FP8)? Seems like 4 bit quantized models would use 1/2 the number of billions of parameters in bytes, because each parameter is half a byte, right?

Yes, it's more a rule of thumb than napkin math I suppose. The difference allows space for the KV cache which scales with both model size and context length, plus other bits and bobs like multimodal encoders which aren't always counted into the nameplate model size.

Re: Qwen3: Think deeper, act faster

#182

Post by Torkel »

markisus wrote: post #174
bcoates wrote: post #144

Earlier quoted context omitted.

Unless I'm missing something holding it hot is pure waste.

Maybe it will help to have a fluid analogy. You have a leaky bucket. What wastes more water, letting all the water leak out and then refilling it from scratch, or keeping it topped up? The answer depends on how bad the leak is vs how long you are required to maintain the bucket level. At least that’s how I interpret this puzzle.

Does it depend though?

The water (heat) leaking out is what you need to add back. As water level drops (hotend cools) the leaking will slow. So any replenishing means more leakage then you are eventually paying for by adding more water (heat) in.

Re: Qwen3: Think deeper, act faster

#183

Post by herdrick »

markisus wrote: post #174
bcoates wrote: post #144

Earlier quoted context omitted.

Unless I'm missing something holding it hot is pure waste.

Maybe it will help to have a fluid analogy. You have a leaky bucket. What wastes more water, letting all the water leak out and then refilling it from scratch, or keeping it topped up? The answer depends on how bad the leak is vs how long you are required to maintain the bucket level. At least that’s how I interpret this puzzle.

No, you should always wait until the last possible moment to refill the leaky bucket, because the less water in the bucket, the slower it leaks, due to reduced pressure.

Re: Qwen3: Think deeper, act faster

#184

Post by Rooki »

natrys wrote: post #158
furyofantares wrote: post #145

Earlier quoted context omitted.

I similarly have a small, simple spatial reasoning problem that only reasoning models get right, and not all of them, and which Qwen3 on max reasoning still gets wrong. > I put a coin in a cup and slam it upside-down on a glass table. I can't see the coin because the cup is over it. I slide a mirror under the table and see heads. What will I see if I take the cup (and the mirror) away?

My first try (omitting chain of thought for brevity): When you remove the cup and the mirror, you will see tails. Here's the breakdown: Setup: The coin is inside an upside-down cup on a glass table. The cup blocks direct view of the coin from above and below (assuming the cup's base is opaque). Mirror Observation: A mirror is slid under the glass table, reflecting the underside of the coin (the side touching the tabl…

Manus just answered this correctly for me in one crack:

Me: I put a coin in a cup and slam it upside-down on a glass table. I can't see the coin because the cup is over it. I slide a mirror under the table and see heads. What will I see if I take the cup (and the mirror) away?

Manus: This is a classic riddle! Let's think it through. You placed a coin in a cup, turned it upside down on a glass table, and then slid a mirror underneath the table. Looking into the mirror from below, you saw 'heads'. Since the mirror reflects the side of the coin facing downwards through the glass table, the 'heads' side must be facing down. Therefore, the side facing up, hidden inside the cup, must be 'tails'. When you remove the cup, you will see the side facing up, which is tails.

Re: Qwen3: Think deeper, act faster

#185

Post by int_19h »

rahimnathwani wrote: post #130
croemer wrote: post #129

Earlier quoted context omitted.

What is the original? The blog post doesn't state the quantization they benchmarked.

This 61GB one: https://ollama.com/library/qwen3:30b-a3b-fp16 You can see it's roughly the same size as the one in the official repo (16 files of 4GB each): https://huggingface.co/Qwen/Qwen3-30B-A3B/tree/main

fp16 is overkill though. 8-bit is the sweet spot before perf degradation starts getting noticeable.

Re: Qwen3: Think deeper, act faster

#187

Post by sroussey »

daemonologist wrote: post #16
sroussey wrote: post #7

Earlier quoted context omitted.

Well, the link to huggingface is broken at the moment.

It's up now: https://huggingface.co/collections/Qwen/qwen3-67dd247413f0e2... The space loads eventually as well; might just be that HF is under a lot of load.

Yep, there now. Do wish they included ONNX though.

Re: Qwen3: Think deeper, act faster

#188

Post by aurareturn »

RandyOrion wrote: post #137

For ultra large MoEs from deepseek and llama 4, fine-tuning on these models is becoming increasingly impossible for hobbyists and local LLM users. Small and dense models are what local people really need. Although benchmaxxing is not good, I still find this release valuable. Thank you Qwen.

  Small and dense models are what local people really need.
Disagreed. Small and dense is dumber and slower for local inferencing. MoEs is what people actually want on local.
Post reply on HN