Live data from Hacker News

QwQ: Alibaba's O1-like reasoning LLM

qwenlm.github.io

181–190 of 435 posts

Re: QwQ: Alibaba's O1-like reasoning LLM

#181
post #74

This one is pretty impressive. I'm running it on my Mac via Ollama - only a 20GB download, tokens spit out pretty fast and my initial prompts have shown some good results. Notes here: https://simonwillison.net/2024/Nov/27/qwq/

What hardware are you able to run this on?

I am running it on a 32G memory mac mini with an M2 Pro using Ollama. It runs fine, faster than I expected. The way it explains plans for solving problems, then proceeding step by step is impressive.

Re: QwQ: Alibaba's O1-like reasoning LLM

#182
post #148

Earlier quoted context omitted.

It’s quite easy to separate out the ccp from the Chinese people, even if the former would rather you didn’t. Chinas people have done many praiseworthy things throughout history. The ccp doesn’t deserve any reflected glory from that. No one should be so naive as to think that a party that is so fearful of free thought, that it would rather massacre its next generation of leaders and hose off their remains into the gut…

This "CCP vs people" model almost always lead to very poor result, to the point that there's no people part anymore: some would just exaggerate and consider CCP has complete control over everything China, so every researcher in China is controlled by CCP and their action may be propaganda, and even researchers in the States are controlled by CCP because they may still have grandpa in China (seriously, WTF?). I fully…

Private entities face challenges from CCP? I don't think this is true as a blanket statement. For example Evergrande did not receive bailouts for their failed investments which checks out with your statement. But at the same time US and EU have been complaining about state subsidies to Chinese electric car makers giving them an unfair advantage. I guess they help sectors which they see as strategically important.

Re: QwQ: Alibaba's O1-like reasoning LLM

#183
post #180

You must use math questions that have never entered the training data set for testing to know whether LLM has real reasoning capabilities. https://venturebeat.com/ai/ais-math-problem-frontiermath-ben...

Of course. I make up my own test problems, but it is likely that the questions and problems that I make up are not totally unique, that is, probably similar to what is in training data. I usually test new models with word problems and programming problems.

Re: QwQ: Alibaba's O1-like reasoning LLM

#184

I asked the classic 'How many of the letter “r” are there in strawberry?' and I got an almost never ending stream of second guesses. The correct answer was ultimately provided but I burned probably 100x more clockcycles than needed. See the response here: https://pastecode.io/s/6uyjstrt

Ha, interesting. FWIW the response I got is much shorter. It second-guessed itself once, considered 2 alternative interpretations of the question, then gave me the correct answer: https://justpaste.it/fqxbf

Re: QwQ: Alibaba's O1-like reasoning LLM

#185
post #58

Earlier quoted context omitted.

I don’t see why they wouldn’t. If you’re China and willing to pour state resources into LLMs, it’s an incredible ROI if they’re adopted. LLMs are black boxes, can be fine tuned to subtly bias responses, censor, or rewrite history. They’re a propaganda dream. No code to point to of obvious interference.

That is a pretty dark view on almost 1/5th of humanity and a nation with a track record of giving the world important innovations: paper making, silk, porcelain, gunpowder and compass to name the few. Not everything has to be around politics.

Also a nation that just used their cargo ship to deliberately cut two undersea cables. But I guess that's not about politics either?

Re: QwQ: Alibaba's O1-like reasoning LLM

#187
post #74

This one is pretty impressive. I'm running it on my Mac via Ollama - only a 20GB download, tokens spit out pretty fast and my initial prompts have shown some good results. Notes here: https://simonwillison.net/2024/Nov/27/qwq/

uhm the pelican SVG is ... not impressive

Re: QwQ: Alibaba's O1-like reasoning LLM

#188

Earlier quoted context omitted.

What hardware are you able to run this on?

Sorry for the random question, I wonder if you know, what's the status of running LLMs non-NVIDIA GPUs nowadays? Are they viable?

I run llama on 7900XT 20GB, works just fine.
Post reply on HN