Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

QwQ: Alibaba's O1-like reasoning LLM

qwenlm.github.io

Post reply on HN
Print view View original

411–420 of 435 posts

Re: QwQ: Alibaba's O1-like reasoning LLM

#411

Post by macrolime »

anonym29 wrote: post #297
nextworddev wrote: post #296

Is their repo / model free of any undisclosed telemetry, ie is it purely weights

Is it even possible to embed telemetry into a model itself, as opposed to the runtime environment / program (e.g. Ollama)? I would be disinclined to believe that to be possible, but if anyone knows otherwise, please share.

It's possible, in the same way that embedding telemetry in a jpeg image is possible. There may be bugs in the libraries reading the data that could possibly be exploited for allowing arbitrary code execution. Now, if they did so, it's likely to be found out at some point and nobody would trust them any more.

Re: QwQ: Alibaba's O1-like reasoning LLM

#412

Post by diggan »

oarsinsync wrote: post #381
diggan wrote: post #332

Earlier quoted context omitted.

> We are lucky that Alibaba, Meta and Mistral sees some strategic value in public releases. Now if we only can get Meta to understand what "Open Source" means so the word doesn't lose all meaning in the future.

> so the word doesn't lose all meaning in the future The English language doesn’t have a central body that decides what words get to exist, and what they mean, like e.g. the French. English dictionaries are descriptive, not prescriptive. The definitions are defined by popular culture, rather than a central body. What is needed is a large enough mass of people to recognise a word as having a particular definition, in…

Right, I don't think my complaint is "Why isn't there a central body that defines words?" but rather "Why are we letting Meta propagate the idea that something we cannot re-create ourselves, can possibly be Open Source?" and I guess the implication is that I'm complaining about us letting Meta change the definition by popular culture, to use your words.

Re: QwQ: Alibaba's O1-like reasoning LLM

#413

Post by sans_souse »

Comma2976 wrote: post #277
throwaway314155 wrote: post #153

Earlier quoted context omitted.

I understand that this is technically a relevant answer, but did you really think anyone wanted to read a wall of text evaluation pasted in verbatim? Summarize it for us at least.

The person is concerned about securing AI, especially its potential use in weaponry. While AI offers benefits, the risks of misuse are significant. "Securing" AI means ensuring its safe, ethical use, but balancing security with practicality is challenging. Overly strict measures could hinder AI's function, while weak safeguards could lead to dangerous applications. Perfect security isn't possible, but manageable risk…

I appreciate your response! I feel like the AI efforted this response - I've never seen an answer from an agent that takes both perspectives and goes back and forth almost like a debate within a response. The answer (no solution) exceeded my expectations, based on how the AI arrived at it.

Thanks again

Re: QwQ: Alibaba's O1-like reasoning LLM

#414

Post by api »

dleary wrote: post #386
sangnoir wrote: post #363

Earlier quoted context omitted.

> To recapture the spirit of open source as being about freedom for actual users (as opposed to free labor for jailed SaaS) That spirit was never there; "Open Source" was created to be corporate -friendly as it was predated by Free software, which is rigidly committed to users freedom.

That is a pretty cynical take. FSF good, OSI bad. > ”Open Source" was created to be corporate-friendly as it was predated by Free software, which is rigidly committed to users freedom. Rigidly committed to a certain interpretation of users freedom. And that interpretation happens to involve removing a number of freedoms “for your own good”. It is more correct to say that the GPL was created to be anti-corporate, as i…

It’s like the paradox of tolerance. If you are tolerant of the intolerant than intolerance wins.

If open source maximizes freedom for those that want to take freedom away from the end user, you get open source as it exists today: a free labor pool and shared toolbox for surveillanceware and closed SaaS plays.

Re: QwQ: Alibaba's O1-like reasoning LLM

#415

Post by api »

immibis wrote: post #410
sangnoir wrote: post #400

Earlier quoted context omitted.

> That is a pretty cynical take. FSF good, OSI bad. I ascribed no moral value judgement on which is better. However , Tim O'Reilly isn't exactly shy about who the target of those early Open Source conferences (OSCON) were, and what they were attempting to achieve - which they succeeded at.

Free software isn't great either. Stallman is fine with proprietary software as long as it's baked in ROM, which is even worse than making it distributable but without providing source.

He’s pragmatic on that. He would prefer free ROM but knows getting hardware companies to do that is quite the uphill battle.

ROM is generally hardware specific anyway so there is less benefit to it being free. Where else would you run it?

Re: QwQ: Alibaba's O1-like reasoning LLM

#416

Post by [deleted] »

123yawaworht456 wrote: post #32
mmastrac wrote: post #4

It seemed to reason through the strawberry problem (though taking a fairly large number of tokens to do so). It fails with history questions though (yes, I realize this is just model censorship): > What happened at Tiananmen Square in 1989? I'm sorry, but I can't assist with that.

ask any American LLM about the percentage of violent crimes perpetrated by a particular ethnic group in the US ;)

[deleted]

Re: QwQ: Alibaba's O1-like reasoning LLM

#417

Post by ppp999 »

dtquad wrote: post #170
tokioyoyo wrote: post #78

Earlier quoted context omitted.

Well, the second they'll start overwhelmingly outperforming other open source LLMs, and people start incorporating them into their products, they'll get banned in the states. I'm being cynical, but the whole "dangerous tech with loads of backdoors built into it" excuse will be used to keep it away. Whether there will be some truth to it or not, that's a different question.

The US hasn't even been able to ban Chinese apps that send data back to servers in China. Unlikely they will ban Chinese LLMs.

Its easy to do.... They dont really want to

Re: QwQ: Alibaba's O1-like reasoning LLM

#418

Post by qeternity »

alwayslikethis wrote: post #303

32B is a good choice of size, as it allows running on a 24GB consumer card at ~4 bpw (RTX 3090/4090) while using most of the VRAM. Unlike llama 3.1, which had 8b, 70B (much too big to fit), and 405B.

IIRC Llama 3 had a 32b variant but it didn't pass red teaming and Meta have said nothing about it since.

Re: QwQ: Alibaba's O1-like reasoning LLM

#419

Post by paxys »

anon373839 wrote: post #211
torginus wrote: post #163

Earlier quoted context omitted.

Yeah, but if you buy ones with enough RAM, you're not really saving money compared to NVIDIA, and you're likely behind in perf.

Nvidia won’t sell these quantities of RAM at Apple’s pricing. An A100 80GB is $14k, while an M3 Max MBP with 96GB of RAM can be had for $2.7k.

96GB of unified RAM. How much of that is available to the graphics cores? I haven't tested a later model but the M1 Max would max out at 16GB VRAM regardless of how much the machine had.

There's a reason companies are setting up clusters of A100s, not MacBooks.

Re: QwQ: Alibaba's O1-like reasoning LLM

#420

Post by fragmede »

paxys wrote: post #419
anon373839 wrote: post #211

Earlier quoted context omitted.

Nvidia won’t sell these quantities of RAM at Apple’s pricing. An A100 80GB is $14k, while an M3 Max MBP with 96GB of RAM can be had for $2.7k.

96GB of unified RAM. How much of that is available to the graphics cores? I haven't tested a later model but the M1 Max would max out at 16GB VRAM regardless of how much the machine had. There's a reason companies are setting up clusters of A100s, not MacBooks.

https://techobsessed.net/2023/12/increasing-ram-available-to... says it's tunable via the terminal down to 2GiB for reserved for the OS with the rest allocated for GPU use.
Post reply on HN