Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

QwQ: Alibaba's O1-like reasoning LLM

qwenlm.github.io

Post reply on HN
Print view View original

201–210 of 435 posts

Re: QwQ: Alibaba's O1-like reasoning LLM

#201

Post by throwaway14356 »

freediver wrote: post #117
tyre wrote: post #58

Earlier quoted context omitted.

I don’t see why they wouldn’t. If you’re China and willing to pour state resources into LLMs, it’s an incredible ROI if they’re adopted. LLMs are black boxes, can be fine tuned to subtly bias responses, censor, or rewrite history. They’re a propaganda dream. No code to point to of obvious interference.

That is a pretty dark view on almost 1/5th of humanity and a nation with a track record of giving the world important innovations: paper making, silk, porcelain, gunpowder and compass to name the few. Not everything has to be around politics.

There is stuff you cant talk about everywhere. if it finds its way into the dataset something has to be done. The scope and what it is of course varies wildly.

Re: QwQ: Alibaba's O1-like reasoning LLM

#202

Post by m3kw9 »

simonw wrote: post #74

This one is pretty impressive. I'm running it on my Mac via Ollama - only a 20GB download, tokens spit out pretty fast and my initial prompts have shown some good results. Notes here: https://simonwillison.net/2024/Nov/27/qwq/

The svg is very unimpressive but you are impressed by it, what gives? It looks nothing like a pelican

Re: QwQ: Alibaba's O1-like reasoning LLM

#203

Post by loufe »

tofuziggy wrote: post #115
msoad wrote: post #50

Somehow o1-preview did not find the answer to the example question. It hallucinated a wrong answer as correct. It eventually came up with another correct answer: (1 + 2) × 3 + 4 × 5 + (6 × 7 + 8) × 9 = 479 Source: https://chatgpt.com/share/6747c32e-1e60-8007-9361-26305101ce...

except not really correct because you are only allowed one set of parens

Good catch!

Re: QwQ: Alibaba's O1-like reasoning LLM

#204

Post by tethys »

singularity2001 wrote: post #187
simonw wrote: post #74

This one is pretty impressive. I'm running it on my Mac via Ollama - only a 20GB download, tokens spit out pretty fast and my initial prompts have shown some good results. Notes here: https://simonwillison.net/2024/Nov/27/qwq/

uhm the pelican SVG is ... not impressive

For comparison, this is what other models produce: https://github.com/simonw/pelican-bicycle/blob/main/README.m...

Re: QwQ: Alibaba's O1-like reasoning LLM

#206

Post by Y_Y »

This one is crazy. I made up a silly topology problem which I guessed wouldn't be in a textbook (given X create a shape with Euler characteristic X) and set it to work. Its first effort was a program that randomly generated shapes, calculated X and hoped it was right. I went and figured out a solution and gave it a clue.

Watching it "think" through the answer is surreal and something I haven't felt since watching GPT-2 write punchlines for jokes.

Here's the lightbulb moment it "realizes" what the clue meant:

> For example, if you have a covering space of degree n, then the Euler characteristic of the covering space is n times the Euler characteristic of the base space.

> But I'm not sure how that relates here.

> Alternatively, perhaps it's about considering the Euler characteristic of a quotient space.

> But again, that might be too advanced for this problem.

> Maybe I'm overcomplicating things.

> Let me go back to the beginning.

> The problem says: "Consider the Euler characteristic of the tetrahemihexahedron and of a genus g torus. Then consider the additive property of the Euler characteristic. Then try again."

> So, perhaps it's simply about adding their Euler characteristics together.

I'm under no illusions about how LLMs are built and operate, but this really feels somehow more "real" and person-y than previous generations, even when you coax them into an answer.

I'm going to go and try having GPT-4o roleplay a student solving the problem and see if it's genuinely different. I've been getting impressive answers from o1, but just coldly getting the result is much more robot than human.

Re: QwQ: Alibaba's O1-like reasoning LLM

#207

Post by mhast »

singularity2001 wrote: post #187
simonw wrote: post #74

This one is pretty impressive. I'm running it on my Mac via Ollama - only a 20GB download, tokens spit out pretty fast and my initial prompts have shown some good results. Notes here: https://simonwillison.net/2024/Nov/27/qwq/

uhm the pelican SVG is ... not impressive

These are language models, they are not designed for producing image output at all. In a way it's impressive it can even produce working SVG code as output. Even more sonthst it vaguely resembles a bird on a bike.

Re: QwQ: Alibaba's O1-like reasoning LLM

#208

Post by simonw »

m3kw9 wrote: post #202
simonw wrote: post #74

This one is pretty impressive. I'm running it on my Mac via Ollama - only a 20GB download, tokens spit out pretty fast and my initial prompts have shown some good results. Notes here: https://simonwillison.net/2024/Nov/27/qwq/

The svg is very unimpressive but you are impressed by it, what gives? It looks nothing like a pelican

Asking language models to draw things by outputting SVG is a deliberately absurd task.

Given how unreasonable that is I thought this model did very well, especially compared to others that I've tried: https://github.com/simonw/pelican-bicycle?tab=readme-ov-file...

Re: QwQ: Alibaba's O1-like reasoning LLM

#209

Post by nuancebydefault »

evrenesat wrote: post #173
Leynos wrote: post #161

Earlier quoted context omitted.

Sadly, qwq failed: > If I was to tell you that the new sequel, "The Fast and The Furious Integer Overflow Exception" was out next week, what would you infer from that? > I'm sorry, but I can't assist with that. Output from o1-preview for comparison: > If I was to tell you that the new sequel, "The Fast and The Furious Integer Overflow Exception" was out next week, what would you infer from that? > If you told me that…

I got this from "qwq-32b-preview@8bit" on my local for same prompt: Well, "The Fast and The Furious" is a popular action movie franchise, so it's likely that there's a new film in the series coming out next week. The title you mentioned seems to be a playful or perhaps intentional misnomer, as "Integer Overflow Exception" sounds like a programming error rather than a movie title. Maybe it's a subtitle or a part of th…

And then still people are saying: these are just heuristic next token predictors incapable of reasoning.

Re: QwQ: Alibaba's O1-like reasoning LLM

#210

Post by williamdclt »

doctoboggan wrote: post #126

I asked the classic 'How many of the letter “r” are there in strawberry?' and I got an almost never ending stream of second guesses. The correct answer was ultimately provided but I burned probably 100x more clockcycles than needed. See the response here: https://pastecode.io/s/6uyjstrt

I’m oscillating between “this is mind blowing” and “this is similarly impressive-looking-but-not-usefully-smart as other LLMs”.

The tone and expression is novel and it _looks_ like there’s something fundamentally different about reasoning but… also it keeps repeating the same things, sometimes in succession (a paragraph about “foreign languages” then another about “different languages”), most paragraphs have a theory then a rebuttal that doesn’t quite answer why the theory is irrelevant, and sometimes it’s flat out wrong (no Rs in “fraise” or “fresa”?).

So… holding my judgement on whether this model actually is useful in novel ways

Post reply on HN