Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

QwQ-32B: Embracing the Power of Reinforcement Learning

qwenlm.github.io

Post reply on HN
Print view View original

71–80 of 178 posts

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#71

Post by hbbio »

antirez wrote: post #39

Note the massive context length (130k tokens). Also because it would be kinda pointless to generate a long CoT without enough context to contain it and the reply. EDIT: Here we are. My first prompt created a CoT so long that it catastrophically forgot the task (but I don't believe I was near 130k -- using ollama with fp16 model). I asked one of my test questions with a coding question totally unrelated to what it say…

"My first prompt created a CoT so long that it catastrophically forgot the task"

Many humans would do that

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#72

Post by zamadatix »

wizee wrote: post #50
antirez wrote: post #39

Note the massive context length (130k tokens). Also because it would be kinda pointless to generate a long CoT without enough context to contain it and the reply. EDIT: Here we are. My first prompt created a CoT so long that it catastrophically forgot the task (but I don't believe I was near 130k -- using ollama with fp16 model). I asked one of my test questions with a coding question totally unrelated to what it say…

Ollama defaults to a context of 2048 regardless of model unless you override it with /set parameter num_ctx [your context length]. This is because long contexts make inference slower. In my experiments, QwQ tends to overthink and question itself a lot and generate massive chains of thought for even simple questions, so I'd recommend setting num_ctx to at least 32768. In my experiments of a couple mechanical engineeri…

I always see:

  /set parameter num_ctx
Explained but never the follow up:

  /save
So you don't have to do the parameter change every load. Is there a better way or is it kind of like setting num_ctx in that "you're just supposed to know"?

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#73

Post by WiSaGaN »

iamronaldo wrote: post #2

This is insane matching deepseek but 20x smaller?

I think it will be more akin to o1-mini/o3-mini instead of r1. It is a very focused reasoning model good at math and code, but probably would not be better than r1 at things like general world knowledge or others.

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#74

Post by rahimnathwani »

daemonologist wrote: post #53

It says "wait" (as in "wait, no, I should do X") so much while reasoning it's almost comical. I also ran into the "catastrophic forgetting" issue that others have reported - it sometimes loses the plot after producing a lot of reasoning tokens. Overall though quite impressive if you're not in a hurry.

Is the model using budget forcing?

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#75

Post by whatshisface »

h0l0cube wrote: post #54
nazgulsenpai wrote: post #11

Earlier quoted context omitted.

If I had to guess, more tariffs and sanctions that increase the competing nation's self-reliance and harm domestic consumers. Perhaps my peabrain just can't comprehend the wisdom of policymakers on the sanctions front, but it just seems like all it does is empower the target long-term.

The tarrifs are for the US to build it's own domestic capabilities, but this will ultimately shift the rest of the world's trade away from the US and toward each other. It's a trade-off – no pun intended – between local jobs/national security and downgrading their own economy/geo-political standing/currency. Anyone who's been making financial bets on business as usual for globalization is going to see a bit of a spee…

Tariffs don't create local jobs, they shut down exporting industries (other countries buy our exports with the dollars we pay them for our imports) and some of those people may over time transition to non-export industries.

Here's an analysis indicating how many jobs would be destroyed in total over several scenarios: https://taxfoundation.org/research/all/federal/trump-tariffs...

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#76

Post by [deleted] »

gagan2020 wrote: post #5

Chinese strategy is open-source software part and earn on robotics part. And, They are already ahead of everyone in that game. These things are pretty interesting as they are developing. What US will do to retain its power? BTW I am Indian and we are not even in the race as country. :(

[deleted]

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#77

Post by h0l0cube »

whatshisface wrote: post #75
h0l0cube wrote: post #54

Earlier quoted context omitted.

The tarrifs are for the US to build it's own domestic capabilities, but this will ultimately shift the rest of the world's trade away from the US and toward each other. It's a trade-off – no pun intended – between local jobs/national security and downgrading their own economy/geo-political standing/currency. Anyone who's been making financial bets on business as usual for globalization is going to see a bit of a spee…

Tariffs don't create local jobs, they shut down exporting industries (other countries buy our exports with the dollars we pay them for our imports) and some of those people may over time transition to non-export industries. Here's an analysis indicating how many jobs would be destroyed in total over several scenarios: https://taxfoundation.org/research/all/federal/trump-tariffs...

They will, out of sheer necessity. Local industries will be incentivized to restart. And of course, there are already carve-outs for the automotive sector that needs steel, overseas components, etc. I expect more carve-outs will be made, esp. for the military.

I don't think the tariffs are being managed intelligently, but they will have the intended effect of moving manufacturing back to the US, even if, in the short term, it's going to inflate prices, and yes, put a lot of businesses in peril.

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#78

Post by lostmsu »

dulakian wrote: post #64
rpastuszak wrote: post #58

Earlier quoted context omitted.

How much vram do you need to run this model? Is 48 gb unified memory enough?

I am using the Q6_K_L quant and it's running at about 40G of vram with the KV cache. Device 1 [NVIDIA GeForce RTX 4090] MEM[||||||||||||||||||20.170Gi/23.988Gi] Device 2 [NVIDIA GeForce RTX 4090] MEM[||||||||||||||||||19.945Gi/23.988Gi]

What's the context length?

Re: QwQ-32B: Embracing the Power of Reinforcement Learning

#79

Post by whatshisface »

h0l0cube wrote: post #77
whatshisface wrote: post #75

Earlier quoted context omitted.

Tariffs don't create local jobs, they shut down exporting industries (other countries buy our exports with the dollars we pay them for our imports) and some of those people may over time transition to non-export industries. Here's an analysis indicating how many jobs would be destroyed in total over several scenarios: https://taxfoundation.org/research/all/federal/trump-tariffs...

They will, out of sheer necessity. Local industries will be incentivized to restart. And of course, there are already carve-outs for the automotive sector that needs steel, overseas components, etc. I expect more carve-outs will be made, esp. for the military. I don't think the tariffs are being managed intelligently, but they will have the intended effect of moving manufacturing back to the US, even if, in the short…

You're thinking about the bolt factory that will open, but what about the factories that close? Putting, as you say, businesses in peril, gets rid of jobs.

As for the carve-outs, there isn't a single US industry that doesn't rely on imports from lower down the supply chain.

Post reply on HN