Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

QwQ: Alibaba's O1-like reasoning LLM

qwenlm.github.io

Post reply on HN
Print view View original

361–370 of 435 posts

Re: QwQ: Alibaba's O1-like reasoning LLM

#361

Post by unsupp0rted »

bofadeez wrote: post #345

It still fails at very simple stuff. E.g. "I put an ordinary rock into a glass of water. I then turn the glass of water upside down, do a little dance, and then turn the glass right side up again. Where is the rock now?" 100+ lines later... "The rock is at the bottom of the glass, submerged in the water." Models from a year ago get this right sometimes https://pastebin.com/em5TT4Zn

Claude 3.5 Sonnet (current):

> The rock would still be at the bottom of the glass. When you turn the glass upside down, the rock falls toward the bottom due to gravity. When you turn it right side up, it falls back to the original bottom. The dance steps don't affect this outcome - gravity consistently pulls the rock toward Earth.

Me:

> Are you certain?

Claude:

> No, I apologize - I jumped to a conclusion. Let me think this through carefully:

> When the glass is turned upside down, water would flow out due to gravity, taking the rock with it. So after completing the sequence, the rock would be on the surface where it fell when the glass was inverted, not in the glass at all.

Re: QwQ: Alibaba's O1-like reasoning LLM

#362

Post by ponector »

seanmcdirmid wrote: post #336
alwayslikethis wrote: post #330

Earlier quoted context omitted.

Macs that can run it are quite a bit more expensive than a 3090. GPUs can also do finetuning and run other models with larger batch sizes which Macs would struggle with. Also, for the models that fit both, an nvidia card can run it much faster.

It’s really not that expensive to get an M4 pro with 64GB. But you really want a Max or ultra, and you aren’t going to be able to do much with images. You are limited on the GPU part. 3090s aren’t sold anymore, so you have to scrounge for them in the used market, 4090s are a couple K? The local LLM scene is still pricey and confusing.

>>not that expensive to get an M4 pro with 64GB

$3900 as the starting price does not sound 'not that expensive' for me.

Re: QwQ: Alibaba's O1-like reasoning LLM

#363

Post by sangnoir »

api wrote: post #360
immibis wrote: post #347

Earlier quoted context omitted.

Words mean whatever's convenient to the bottom line, which is why the OSI (a consortium of Amazon, Google, Microsoft etc) still doesn't recognize the SSPL, as it would be particularly inconvenient for clouds.

So you're saying open source doesn't exist to be free labor for SaaS? The OSI is fully captured by companies with a vested interest in promoting that model and/or using open source to 'dump' on the market and commoditize their compliments. To recapture the spirit of open source as being about freedom for actual users (as opposed to free labor for jailed SaaS) and a mutualistic gift culture (as opposed to a take-take-…

> To recapture the spirit of open source as being about freedom for actual users (as opposed to free labor for jailed SaaS)

That spirit was never there; "Open Source" was created to be corporate -friendly as it was predated by Free software, which is rigidly committed to users freedom.

Re: QwQ: Alibaba's O1-like reasoning LLM

#364

Post by torginus »

itsoktocry wrote: post #337
torginus wrote: post #292

Earlier quoted context omitted.

I think the true edge of CoT models will come from layman usability. While I can easily prompt Claude for examples and then manually modify the code to fill in the gaps, general domain knowledge and technical understanding is absolutely required from the human sitting in front of the screen. With o1, a layman can sit in front of the computer, and ask 'I want a website for tracking deliveries for my webshop and make i…

> With o1, a layman can sit in front of the computer, and ask 'I want a website for tracking deliveries for my webshop and make it pretty', and the model will do it. I just punched that prompt into Sonnet 3.5 and o1 and I wouldn't say that o1 is doing anything better than Sonnet. o1 certainly didn't "do it", it gave me a very broad outline of how to accomplish that, from "Define requirements" to "Test and deply on Ve…

Honestly I had pretty good success with it.

I wanted to try AWS batch for an example app after people here suggested it, and I had something running with like 2 prompts.

Re: QwQ: Alibaba's O1-like reasoning LLM

#365

Post by kaba0 »

icoder wrote: post #266
fngjdflmdflg wrote: post #231

It gets the Sally question correct, but it takes more than 100 lines of reasoning. >Sally has three brothers. Each brother has two sisters. How many sisters does sally have? Here is the answer: https://pastebin.com/JP2V92Kh

This line had me laughing: "But that doesn't make sense because Sally can't be her own sister." Having said this, how many 'lines' of reasoning does the average human need? It's a weird comparison perhaps but the point is does it really matter if it needs 100 or 100k 'lines', if it could hide that (just as we hide our thoughts or even can't really access the - semi-parallel - things our brain does to come to an answe…

The implementation details don't matter. LLMs not being able to properly reason though is a fundamental limitation and no amount of re-running will help.

Re: QwQ: Alibaba's O1-like reasoning LLM

#366

Post by seanmcdirmid »

ponector wrote: post #362
seanmcdirmid wrote: post #336

Earlier quoted context omitted.

It’s really not that expensive to get an M4 pro with 64GB. But you really want a Max or ultra, and you aren’t going to be able to do much with images. You are limited on the GPU part. 3090s aren’t sold anymore, so you have to scrounge for them in the used market, 4090s are a couple K? The local LLM scene is still pricey and confusing.

>>not that expensive to get an M4 pro with 64GB $3900 as the starting price does not sound 'not that expensive' for me.

M4 Pro with 64GB of ram starts at $3900? It doesn't take that much searching to know that is BS:

https://www.apple.com/shop/buy-mac/mac-mini/apple-m4-pro-chi...

An upgraded M4 Pro mac mini to 64GB is $1999. You'll probably want the extra GPUs, though, bringing you up to $2199. But you probably want the max (not offered for the mini) or ultra to do serious LLM work. An M4 Max in a 14 inch MBP with 64GB of ram will cost $3900, but that's a whole step up from the M4 pro. I'm waiting to see what they do with the Mac Studio refresh (I don't need a max in a portable, and I'm betting I can get an M4 Ultra with 64GB ram for $4000 or so).

Re: QwQ: Alibaba's O1-like reasoning LLM

#367

Post by pton_xd »

delusional wrote: post #343
elorant wrote: post #335

Earlier quoted context omitted.

Care to say how many tokens per second you're getting?

For the "How many r's in the word strawberry" total duration: 15.278476756s load duration: 14.982999ms prompt eval count: 47 token(s) prompt eval duration: 5ms prompt eval rate: 9400.00 tokens/s eval count: 377 token(s) eval duration: 15.257s eval rate: 24.71 tokens/s

15 seconds to count the number of r's in a string using 120 TFlops... Really puts into perspective that we've got a long way yet to go.

Re: QwQ: Alibaba's O1-like reasoning LLM

#368

Post by kaba0 »

exact_string wrote: post #214
sysmax wrote: post #132

Earlier quoted context omitted.

Well, to be perfectly honest, it's hard question for an LLM that reasons in tokens and not letters. Reminds me of that classic test that kids easily pass and grownups utterly fail. The test looks like this: continue a sequence: 0 - 1 5 - 0 6 - 1 7 - 0 8 - 2 9 - ? Grownups try to find a pattern in the numbers, different types of series, progressions, etc. The correct answer is 1 because it's the number of circles in t…

These tests always make me wonder: What qualifies as a valid pattern rule? For example, why wouldn't "0" be a correct answer here (rule being "every other number on the right should be 0, other numbers do not have a pattern")?

This is exactly what I hated about these kind of questions!!

Or just, the pattern simply repeats for the n next number.. like come on, there is no objective metric for which rule is "better".

Re: QwQ: Alibaba's O1-like reasoning LLM

#369

Post by FooBarWidget »

imdsm wrote: post #316
echelon wrote: post #315

Earlier quoted context omitted.

Most of the papers in machine learning are coming from China. The vast majority. Most of the open source models on GitHub, too. Hailuo, Kling, Vidu, and Hunyuan (posted on Banodoko) blow Sora and Runway out of the water. China is dominating at this field. And if they begin releasing weights as open source, that'll mean foundation model companies can only bank on the thin facade of product. That's a really good strate…

Are they making breakthroughs or are they taking what exists and copying/marginally improving?

"taking what exists and marginally improving" You just described most scientific progress. Real breakthroughs only happen in the newspapers and movies.

Re: QwQ: Alibaba's O1-like reasoning LLM

#370

Post by gscott »

MaxPock wrote: post #338
TrainedMonkey wrote: post #312

So western controls on training hardware (hello NVIDIA) seem to have failed. I wonder if there will be any repercussions here.

Btw renting GPUs in Nvidia GPUs in China is cheaper than in the US

Crypto mining was made illegal in China, likely leaves a lot of GPU's available for AI.
Post reply on HN