QwQ-32B: Embracing the Power of Reinforcement Learning • Hacker News Board

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#41

Post by rvz » Wed, Mar 05, 2025, 10:48 PM UTC

The AI race to zero continues to accelerate with downloadable free AI models which have already won the race and destroying closed source frontier AI models.

They are once again getting squeezed in the middle and this is even before Meta releases Llama 4.

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#42

Post by arnaudsm » Wed, Mar 05, 2025, 10:56 PM UTC

esafak wrote: post #33

bearjaws wrote: post #14

Available on ollama now as well.

Is this the best way to run your own models these days?

It's the easiest to setup, but you can get 2x-6x faster with TGI and vLLM depending on the scenario.

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#43

Post by arnaudsm » Wed, Mar 05, 2025, 10:57 PM UTC

bearjaws wrote: post #14

Available on ollama now as well.

https://ollama.com/library/qwq:32b

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#44

Post by dr_dshiv » Wed, Mar 05, 2025, 11:00 PM UTC

I love that emphasizing math learning and coding leads to general reasoning skills. Probably works the same in humans, too.

20x smaller than Deep Seek! How small can these go? What kind of hardware can run this?

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#45

Post by ignorantguy » Wed, Mar 05, 2025, 11:10 PM UTC

antirez wrote: post #39

Note the massive context length (130k tokens). Also because it would be kinda pointless to generate a long CoT without enough context to contain it and the reply. EDIT: Here we are. My first prompt created a CoT so long that it catastrophically forgot the task (but I don't believe I was near 130k -- using ollama with fp16 model). I asked one of my test questions with a coding question totally unrelated to what it say…

Yeah it did the same in my case too. it did all the work in the tokens. but did not spit out the actual answer. I was not even close to 100K tokens

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#46

Post by UncleOxidant » Wed, Mar 05, 2025, 11:24 PM UTC

manmal wrote: post #31

I guess I won’t be needing that 512GB M3 Ultra after all.

I think the Framework AI PC will run this quite nicely.

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#47

Post by UncleOxidant » Wed, Mar 05, 2025, 11:29 PM UTC

holoduke wrote: post #34

fsndz wrote: post #29

Earlier quoted context omitted.

super impressive. we won't need that many GPUs in the future if we can have the performance of DeepSeek R1 with even less parameters. NVIDIA is in trouble. We are moving towards a world of very cheap compute: https://medium.com/thoughts-on-machine-learning/a-future-of-...

Have you heard of Jevons paradox? That says that whenever new tech is used to make something more efficient the tech is just upscaled to make the product quality higher. Same here. Deepseek has some algoritmic improvements that reduces resources for the same output quality. But increasig resources (which are available) will increase the quality. There will be always need for more compute. Nvidia is not in trouble. Th…

I agree that the Jevons paradox can apply here, however, there have been several "breakthroughs" in the last couple of months (R1, diffusion LLMs, this) that really push the amount of GPU compute down such that I think it's going to be problematic for companies that went out and bought boatloads of GPUs (like OpenAI, for example). So while it might not be bad news for NVidia (given Jevons) it does seem to be bad news for OpenAI.

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#48

Post by mark_l_watson » Wed, Mar 05, 2025, 11:30 PM UTC

neither_color wrote: post #40

rspoerri wrote: post #15

Earlier quoted context omitted.

i could not find it, where did you?

ollama pull qwq

I am running ‘ollama run qwq’ - same thing.

Sometimes I feel like forgetting about the best commercial models and just use the olen weights models. I am retired so I don’t need state of the art.

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#49

Post by samstave » Wed, Mar 05, 2025, 11:34 PM UTC

dr_dshiv wrote: post #44

I love that emphasizing math learning and coding leads to general reasoning skills. Probably works the same in humans, too. 20x smaller than Deep Seek! How small can these go? What kind of hardware can run this?

>I love that emphasizing math learning and coding leads to general reasoning skills

Its only logical.

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#50

Post by wizee » Wed, Mar 05, 2025, 11:36 PM UTC

antirez wrote: post #39

Note the massive context length (130k tokens). Also because it would be kinda pointless to generate a long CoT without enough context to contain it and the reply. EDIT: Here we are. My first prompt created a CoT so long that it catastrophically forgot the task (but I don't believe I was near 130k -- using ollama with fp16 model). I asked one of my test questions with a coding question totally unrelated to what it say…

Ollama defaults to a context of 2048 regardless of model unless you override it with /set parameter num_ctx [your context length]. This is because long contexts make inference slower. In my experiments, QwQ tends to overthink and question itself a lot and generate massive chains of thought for even simple questions, so I'd recommend setting num_ctx to at least 32768.

In my experiments of a couple mechanical engineering problems, it did fairly well in final answers, correctly solving mechanical engineering problems that even DeepSeek r1 (full size) and GPT 4o did wrong in my tests. However, the chain of thought was absurdly long, convoluted, circular, and all over the place. This also made it very slow, maybe 30x slower than comparably sized non-thinking models.

I used a num_ctx of 32768, top_k of 30, temperature of 0.6, and top_p of 0.95. These parameters (other than context length) were recommended by the developers on Hugging Face.