Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Qwen3: Think deeper, act faster

qwenlm.github.io

Post reply on HN
Print view View original

101–110 of 412 posts

Re: Qwen3: Think deeper, act faster

#101

Post by mmoskal »

jasonjmcghee wrote: post #18
minimaxir wrote: post #13

A 0.6B LLM with a 32k context window is interesting, even if it was trained using only distillation (which is not ideal as it misses nuance). That would be a fun base model for fine-tuning. Out of all the Qwen3 models on Hugging Face, it's the most downloaded/hearted. https://huggingface.co/collections/Qwen/qwen3-67dd247413f0e2...

these 0.5 and 0.6B models etc. are _fantastic_ for using as a draft model in speculative decoding. lm studio makes this super easy to do - i have it on like every model i play with now my concern on these models though unfortunately is it seems like architectures very a bit so idk how it'll work

Spec decoding only depends on the tokenizer used. It's transfering either the draft token sequence or at most draft logits to the main model.

Re: Qwen3: Think deeper, act faster

#102

Post by phonon »

stavros wrote: post #98

I have a small physics-based problem I pose to LLMs. It's tricky for humans as well, and all LLMs I've tried (GPT o3, Claude 3.7, Gemini 2.5 Pro) fail to answer correctly. If I ask them to explain their answer, they do get it eventually, but none get it right the first time. Qwen3 with max thinking got it even more wrong than the rest, for what it's worth.

Qwen3-235B-A22B?

Re: Qwen3: Think deeper, act faster

#103

Post by chillfox »

ramesh31 wrote: post #60

Gotta love how Claude is always conventiently left out of all of these benchmark lists. Anthropic really is in a league of their own right now.

Yeah, just a shame their API is consistently overloaded to the point of being useless most of the time (from about midday till late for me).

Re: Qwen3: Think deeper, act faster

#104

Post by stavros »

phonon wrote: post #102
stavros wrote: post #98

I have a small physics-based problem I pose to LLMs. It's tricky for humans as well, and all LLMs I've tried (GPT o3, Claude 3.7, Gemini 2.5 Pro) fail to answer correctly. If I ask them to explain their answer, they do get it eventually, but none get it right the first time. Qwen3 with max thinking got it even more wrong than the rest, for what it's worth.

Qwen3-235B-A22B?

Yep, on Qwen chat.

Re: Qwen3: Think deeper, act faster

#105

Post by stavros »

kenjackson wrote: post #99
stavros wrote: post #98

I have a small physics-based problem I pose to LLMs. It's tricky for humans as well, and all LLMs I've tried (GPT o3, Claude 3.7, Gemini 2.5 Pro) fail to answer correctly. If I ask them to explain their answer, they do get it eventually, but none get it right the first time. Qwen3 with max thinking got it even more wrong than the rest, for what it's worth.

You really had me until the last half of the last sentence.

The plural of anecdote is data.

Re: Qwen3: Think deeper, act faster

#106

Post by omneity »

Excellent release by the Qwen team as always. Pretty much the best open-weights model line so far.

In my early tests however, several of the advertised languages are not really well supported and the model is outputting something that only barely resembles them.

Probably a dataset quality issue for low-resource languages that they cannot personally check for, despite the “119 languages and dialects” claim.

Re: Qwen3: Think deeper, act faster

#107

Post by jasonjmcghee »

mmoskal wrote: post #101
jasonjmcghee wrote: post #18

Earlier quoted context omitted.

these 0.5 and 0.6B models etc. are _fantastic_ for using as a draft model in speculative decoding. lm studio makes this super easy to do - i have it on like every model i play with now my concern on these models though unfortunately is it seems like architectures very a bit so idk how it'll work

Spec decoding only depends on the tokenizer used. It's transfering either the draft token sequence or at most draft logits to the main model.

I suppose that makes sense, for some reason I was under the impression that the models need to be aligned / have the same tuning or they'd have different probability distributions and would reject the draft model really often.

Re: Qwen3: Think deeper, act faster

#108

Post by brigade »

refulgentis wrote: post #61
GodelNumbering wrote: post #48

Earlier quoted context omitted.

There are a lot of variables here such as your hardware's memory bandwidth, speed at which at processes tensors etc. A basic thing to remember: Any given dense model would require X GB of memory at 8-bit quantization, where X is the number of params (of course I am simplifying a little by not counting context size). Quantization is just 'precision' of the model, 8-bit generally works really well. Generally speaking,…

4 bit is absolutely fine . I know this is crazy to here because the big iron folks still debate 16 vs 32 and 8 vs 16 is near verboten in public conversation. I contribute to llama.cpp and have seen many many efforts to measure evaluation perf of various quants, and no matter which way it was sliced (ranging from subjective volunteers doing A/B voting on responses over months, to objective object perplexity loss) Q4 i…

It's incredibly niche, but Gemma 3 27b can recognize a number of popular video game characters even in novel fanart (I was a little surprised at that when messing around with its vision). But the Q4 quants, even with QAT, are very likely to name a random wrong character from within the same franchise, even when Q8 quants name the correct character.

Niche of a niche, but just kind of interesting how the quantization jostles the name recall.

Re: Qwen3: Think deeper, act faster

#109

Post by arthurcolle »

stavros wrote: post #98

I have a small physics-based problem I pose to LLMs. It's tricky for humans as well, and all LLMs I've tried (GPT o3, Claude 3.7, Gemini 2.5 Pro) fail to answer correctly. If I ask them to explain their answer, they do get it eventually, but none get it right the first time. Qwen3 with max thinking got it even more wrong than the rest, for what it's worth.

Hi, I'm starting an evals company, would love to have you as an advisor!

Re: Qwen3: Think deeper, act faster

#110

Post by [deleted] »

stavros wrote: post #98

I have a small physics-based problem I pose to LLMs. It's tricky for humans as well, and all LLMs I've tried (GPT o3, Claude 3.7, Gemini 2.5 Pro) fail to answer correctly. If I ask them to explain their answer, they do get it eventually, but none get it right the first time. Qwen3 with max thinking got it even more wrong than the rest, for what it's worth.

[deleted]
Post reply on HN