Live data from Hacker News

QwQ: Alibaba's O1-like reasoning LLM

qwenlm.github.io

241–250 of 435 posts

Re: QwQ: Alibaba's O1-like reasoning LLM

#241
post #191

Earlier quoted context omitted.

I am running it on a 32G memory mac mini with an M2 Pro using Ollama. It runs fine, faster than I expected. The way it explains plans for solving problems, then proceeding step by step is impressive.

How many tokens per second?

8.4 tps on M1 Pro chip with 32GM RAM (Q4 model, 18GB).

Re: QwQ: Alibaba's O1-like reasoning LLM

#242

Interestingly, it failed today's NY Times Connections while 01-preview nailed it. The prompt if anyone wants to try it: • ENDEAVOR • CURB • NATIONAL • BOARDWALK • HERTZ • TWIN • MOLE • ENTERPRISE • SILICON • PROJECT • TIGER • VOLT • GAME • RAY • SECOND • VENTURE Its a game of NY Times connections. You need to make 4 groups of 4 words. Can you do it?

False start with car rental companies, only three, not four. Hertz has to be a unit then, making the first group hertz, second, mole, and volt.

Similarly, enterprise must be in the business sense. Enterprise, project, venture, endeavor.

Tiger, Ray, National, Twin are all singular versions of baseball teams?

Curb, silicon, boardwalk, and game are left. Boardwalk is the most valuable monopoly property, silicon is the modern equivalent? No clue, maybe I’m missing a cultural reference? What if curb is a verb? Curb your enthusiasm, first word of tv shows? Silicon Valley, game of thrones, maybe there is a show that starts with boardwalk? Seems tenuous.

Edit: yes, there is. Boardwalk Empire. Never watched any of them, but some googling tends me they’re all hbo shows - I knew that about Thrones. That seems slightly more realistic a connection than just first word of tv shows.

Seems like exactly the kind of thing AI would/could theoretically best humans in.

Re: QwQ: Alibaba's O1-like reasoning LLM

#243
post #132

Earlier quoted context omitted.

Well, to be perfectly honest, it's hard question for an LLM that reasons in tokens and not letters. Reminds me of that classic test that kids easily pass and grownups utterly fail. The test looks like this: continue a sequence: 0 - 1 5 - 0 6 - 1 7 - 0 8 - 2 9 - ? Grownups try to find a pattern in the numbers, different types of series, progressions, etc. The correct answer is 1 because it's the number of circles in t…

Damn I guessed the answer to be 9...

88880 = 9

Re: QwQ: Alibaba's O1-like reasoning LLM

#244
I remember having a hard time solving this in my first year of college. This is really impressive.

So I have this problem here: given that for positive integers m and n, 0 ≤ x[m + n] ≤ x[m] + x[n], I need to prove that the limit of x[n]/n exists. Hmm, okay, let's see how to approach this.

Alternative approach: maybe I can show that a_n is convergent by showing that it's a Cauchy sequence.

But that might be complicated.

Alternatively, perhaps I can use the subadditivity to bound a_n.

Wait, let's consider the following:

Since x is subadditive, x[m + n] ≤ x[m] + x[n].

Let me set m = n, then x[2n] ≤ x[n] + x[n] = 2x[n].

Therefore, x[2n]/(2n) ≤ x[n]/n.

So, a_{2n} ≤ a_n.

That means that a_{2n} ≤ a_n.

Similarly, x[3n] ≤ x[2n] + x[n] ≤ 2x[n] + x[n] = 3x[n].

So, x[3n]/(3n) ≤ x[n]/n.

Similarly, x[4n]/(4n) ≤ x[2n]/(2n) ≤ x[n]/n.

It seems like a_{kn} ≤ a_n for any integer k ≥ 1.

This suggests that as n increases, a_n tends to decrease or stay the same.

But wait, in the example where x[n] = n, a_n = 1 for all n, so it's constant.

In the example where x[n] = sqrt(n), a_n = 1/sqrt(n), which decreases to 0.

So, in some cases, a_n is decreasing, and in some cases, it's constant.

But in general, a_n is bounded below by 0 and is decreasing along powers of 2, for example.

Wait, from x[2n]/(2n) ≤ x[n]/n, and x[4n]/(4n) ≤ x[2n]/(2n), and so on, so along the sequence n, 2n, 4n, 8n, etc., a_n is decreasing.

Maybe I can use this to show that a_n has a limit.

Let me consider n = 2^k, for integer k.

Let b_k = x[2^k]/2^k.

Then, from x[2^{k+1}] ≤ x[2^k] + x[2^k] = 2x[2^k], so b_{k+1} = x[2^{k+1}]/2^{k+1} ≤ (2x[2^k])/2^{k+1} = x[2^k]/2^k = b_k.

So, b_{k+1} ≤ b_k, meaning that b_k is a decreasing sequence.

Since b_k ≥ 0, it must converge to some limit, say L.

Now, I need to show that x[n]/n approaches L as n approaches infinity.

Given that b_k is decreasing and converges to L, and for general n, a_n is bounded below by L, and perhaps approaches L.

Wait, more carefully:

For any n, there exists k such that 2^{k-1} Then, x[n]/n ≤ x[2^k]/2^{k-1} = 2(x[2^k]/2^k) = 2b_k.

Since b_k approaches L, for large k, b_k is close to L.

But I need to show that x[n]/n is close to L.

Wait, perhaps I can bound x[n]/n between b_k and something else.

Given that 2^{k-1} x[2^{k-1}].

Therefore, x[n]/n ≤ (2x[2^{k-1}])/n ≤ (2x[2^{k-1}])/2^{k-1} = 2(x[2^{k-1}]/2^{k-1}) = 2b_{k-1}.

But b_{k-1} approaches L as k increases.

Similarly, since n > 2^{k-1}, x[n]/n > x[2^k]/2^k = b_k.

But b_k ≤ b_{k-1}, and both approach L.

Therefore, x[n]/n is squeezed between b_k and 2b_{k-1}, both of which approach L.

Therefore, x[n]/n approaches L as n approaches infinity.

This suggests that the limit exists and is equal to L.

Therefore, the limit of x[n]/n exists.

Alternatively, if I recall correctly, this is exactly what Fekete's lemma states.

So, in conclusion, the limit of x[n]/n exists.

*Final Answer*

\boxed{\lim_{n \to \infty} \frac{x[n]}{n} \text{ exists}}

Re: QwQ: Alibaba's O1-like reasoning LLM

#245

It gets the Sally question correct, but it takes more than 100 lines of reasoning. >Sally has three brothers. Each brother has two sisters. How many sisters does sally have? Here is the answer: https://pastebin.com/JP2V92Kh

I don't think this question is super hard. ChatGPT 4o mini gets this one correct consistently without being asked to reason step by step.

Re: QwQ: Alibaba's O1-like reasoning LLM

#246
post #132

Earlier quoted context omitted.

Well, to be perfectly honest, it's hard question for an LLM that reasons in tokens and not letters. Reminds me of that classic test that kids easily pass and grownups utterly fail. The test looks like this: continue a sequence: 0 - 1 5 - 0 6 - 1 7 - 0 8 - 2 9 - ? Grownups try to find a pattern in the numbers, different types of series, progressions, etc. The correct answer is 1 because it's the number of circles in t…

These tests always make me wonder: What qualifies as a valid pattern rule? For example, why wouldn't "0" be a correct answer here (rule being "every other number on the right should be 0, other numbers do not have a pattern")?

I think it's better phrased as "find the best rule", with a tacit understanding that people mostly agree on what makes a rule decent vs. terrible (maybe not on what makes one great) and a tacit promise that the sequence presented has at least one decent rule and does not have multiple.

A rule being "good" is largely about simplicity, which is also essentially the trick that deep learning uses to escape no-free-lunch theorems.

Re: QwQ: Alibaba's O1-like reasoning LLM

#247
post #74

This one is pretty impressive. I'm running it on my Mac via Ollama - only a 20GB download, tokens spit out pretty fast and my initial prompts have shown some good results. Notes here: https://simonwillison.net/2024/Nov/27/qwq/

uhm the pelican SVG is ... not impressive

You try writing an SVG by hand without being able to look at the render.

Re: QwQ: Alibaba's O1-like reasoning LLM

#248

It gets the Sally question correct, but it takes more than 100 lines of reasoning. >Sally has three brothers. Each brother has two sisters. How many sisters does sally have? Here is the answer: https://pastebin.com/JP2V92Kh

I should have read the blog post. This is a known issue:

>Recursive Reasoning Loops: The model may enter circular reasoning patterns, leading to lengthy responses without a conclusive answer.

So the 100 lines was probably not necessary.

Re: QwQ: Alibaba's O1-like reasoning LLM

#249
post #233

Earlier quoted context omitted.

I haven’t ran QWQ yet, but it’s a 32B. So about 20GB RAM with Q4 quant. Closer to 25GB for the 4_K_M one. You can wait for a day or so for the quantized GGUFs to show up (we should see the Q4 in the next hour or so). I personally use Ollama on an MacBook Pro. It usually takes a day or two for it to show up. Any M series MacBook with 32GB+ of RAM will run this.

> 20GB RAM with Q4 quant. Closer to 25GB for the 4_K_M one how does this math work? are there rules of thumb that you guys know that the rest of us dont?

As a quick estimation, the size of q4 quantized model usually be around 60-70% of the model's parameter. You can preciselly check the quantized model size from .gguf files hosted in huggingface.

Re: QwQ: Alibaba's O1-like reasoning LLM

#250

Earlier quoted context omitted.

The ship was not driven by China, the media reported it incorrectly first.

Do you have a source more recent than https://archive.is/3weox (WSJ article)? It appears to be a Chinese ship, although it is not clear that the Chinese government sanctioned whatever happened.

If you read the article it even states that it's a Chinese ship but with a Russian crew that departed from Russia. They leased it from China. If you have an accident with a leased Chinese car, no one would say "the Chinese did it".
Post reply on HN