Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3: Think deeper, act faster

qwenlm.github.io

Post reply on HN
Print view View original

381–390 of 412 posts

Re: Qwen3: Think deeper, act faster

#381

Post by Havoc »

jasonjmcghee wrote: post #18
minimaxir wrote: post #13

A 0.6B LLM with a 32k context window is interesting, even if it was trained using only distillation (which is not ideal as it misses nuance). That would be a fun base model for fine-tuning. Out of all the Qwen3 models on Hugging Face, it's the most downloaded/hearted. https://huggingface.co/collections/Qwen/qwen3-67dd247413f0e2...

these 0.5 and 0.6B models etc. are _fantastic_ for using as a draft model in speculative decoding. lm studio makes this super easy to do - i have it on like every model i play with now my concern on these models though unfortunately is it seems like architectures very a bit so idk how it'll work

Have you had any luck getting actual speedups? All the combinations I've tried (smallest 0.6 + largest I can fit into 24gb)...all got me slowdowns despite decent hitrate

Re: Qwen3: Think deeper, act faster

#382

Post by [deleted] »

rwj wrote: post #119
stavros wrote: post #116

Earlier quoted context omitted.

Except, very literally, data is a collection of single points (ie what we call "anecdotes").

Except that the plural of anecdotes is definitely not data, because without controlling for confounding variables and sampling biases, you will get garbage.

[deleted]

Re: Qwen3: Think deeper, act faster

#383

Post by [deleted] »

rwj wrote: post #119
stavros wrote: post #116

Earlier quoted context omitted.

Except, very literally, data is a collection of single points (ie what we call "anecdotes").

Except that the plural of anecdotes is definitely not data, because without controlling for confounding variables and sampling biases, you will get garbage.

[deleted]

Re: Qwen3: Think deeper, act faster

#384

Post by absolutelastone »

stavros wrote: post #116
rtaylorgarlock wrote: post #113

Earlier quoted context omitted.

Only in the same way that the plural of 'opinion' is 'fact' ;)

Except, very literally, data is a collection of single points (ie what we call "anecdotes").

one point is a collection of size 1. It is always data.

Re: Qwen3: Think deeper, act faster

#385

Post by 9rx »

stavros wrote: post #116
rtaylorgarlock wrote: post #113

Earlier quoted context omitted.

Only in the same way that the plural of 'opinion' is 'fact' ;)

Except, very literally, data is a collection of single points (ie what we call "anecdotes").

Technically we call it a datum. An anecdote is a story, not a point.

But it is true that colloquially anecdote is sometimes used in place of datum.

Re: Qwen3: Think deeper, act faster

#387

Post by CMay »

Havoc wrote: post #380
CMay wrote: post #378

The Qwen3 32B dense model just fails for me due to a template issue, but the Qwen3 30B A3B model does work. I think the more dense the model is at a small number of active parameters, the more sensitive a model can be to quantization. Only have 24GB of VRAM and have been using a 4-bit quantization which I use for most models. Qwen3 30B A3B is quick since it's MoE, but the results are just unreliable. It uses up all o…

>The Qwen3 32B dense model just fails for me due to a template issue, You're likely either using an old one or the broken on (repo second state). Try the ones unsloth uploaded Definitely works for me on LM studio

Think I tried both the Unsloth and original yesterday, but it looks like the model got updated today so I'm downloading the new version. We'll see how that goes!

Re: Qwen3: Think deeper, act faster

#388

Post by yencabulator »

sega_sai wrote: post #37

With all the different open-weight models appearing, is there some way of figuring out what model would work with sensible speed (> X tok/s) on a standard desktop GPU ? I.e. I have Quadro RTX 4000 with 8G vram and seeing all the models https://ollama.com/search here with all the different sizes, I am absolutely at loss which models with which sizes would be fast enough. I.e. there is no point of me downloading the la…

Well, deepseek-r1:7b on AMD CPU only is ~12 token/s, gemma3:27b-it-qat is ~2.2 token/s. That's pure CPU at about 0.1x of a $3,500 Apple laptop at about 0.1x of the price. It's more a question about your patience, use case, and budget.

For discrete GPUs, RAM size is a harder cutoff. You either can run a model, or you can't.

Re: Qwen3: Think deeper, act faster

#389

Post by bcoates »

markisus wrote: post #377
Torkel wrote: post #182

Earlier quoted context omitted.

Does it depend though? The water (heat) leaking out is what you need to add back. As water level drops (hotend cools) the leaking will slow. So any replenishing means more leakage then you are eventually paying for by adding more water (heat) in.

You can stipulate conditions to make the solution work out in either direction. Suppose the bucket is the size of lake, and the leak is so miniscule that it takes many centuries to detect any loss. And also I need to keep the bucket full for a microsecond. In this case it is better to keep the bucket full, than to let it drain. Now suppose the bucket is made out of chain-link and any water you put into it immediately…

All heat that is lost must be replaced (we must input enough heat that the device returns to T_initial)

Hotter objects lose heat faster, so the longer we delay restoring temperature (for a fixed resume time) the less heat is lost that will need replacement.

Hotter objects require more energy to add another unit of heat, so the cooler we allow the device to get before re-heating (again, resume time is fixed) the more efficient our heating can be.

There is no countervailing effect to balance, preemptive heating of a device before the last possible moment is pure waste no matter the conditions (although the amount of waste will vary a lot, it will always be a positive number)

Even turning the heater off for a millisecond is a net gain.

Re: Qwen3: Think deeper, act faster

#390

Post by LegionMammal978 »

fhd2 wrote: post #245
rwj wrote: post #119

Earlier quoted context omitted.

Except that the plural of anecdotes is definitely not data, because without controlling for confounding variables and sampling biases, you will get garbage.

Based on my limited understanding of analytics, the data set can be full of biases and anomalies, as long as you find a way to account for them in the analysis, no?

The accuracy of your analysis becomes limited to the accuracy of how well you correct for the biases. And it's difficult to measure the bias accurately without lots of good data or cross-examination.
Post reply on HN