Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

QwQ: Alibaba's O1-like reasoning LLM

qwenlm.github.io

Post reply on HN
Print view View original

301–310 of 435 posts

Re: QwQ: Alibaba's O1-like reasoning LLM

#301

Post by teaearlgraycold »

whimsicalism wrote: post #289
JKCalhoun wrote: post #285

Is Alibaba's LLM the "Chinese LLM"? It would appear to have been a U.S.-only game until now. As Eric Schmidt said in the YouTube lecture (that keeps getting pulled down), LLM's have been a rich-companies game.

you only think that because you haven’t been paying close attention qwen, deepseek, yi - there have been a number of high quality, open chinese competitors

And AI21, which is Israeli

Re: QwQ: Alibaba's O1-like reasoning LLM

#304

Post by coconut08 »

Terretta wrote: post #193
syntaxing wrote: post #44

Earlier quoted context omitted.

I haven’t ran QWQ yet, but it’s a 32B. So about 20GB RAM with Q4 quant. Closer to 25GB for the 4_K_M one. You can wait for a day or so for the quantized GGUFs to show up (we should see the Q4 in the next hour or so). I personally use Ollama on an MacBook Pro. It usually takes a day or two for it to show up. Any M series MacBook with 32GB+ of RAM will run this.

On Macbooks with Apple Silicon consider MLX models from MLX community: https://huggingface.co/collections/mlx-community/qwq-32b-pre... For a GUI, LM Studio 0.3.x is iterating MLX support: https://lmstudio.ai/beta-releases When searching in LM Studio, you can narrow search to the mlx-community.

on macos with lm-studio is it better to use the mlx-community releases over the one that lm-studio releases?

also I didn't install a beta and mine says i'm using 3.5 which is what the beta also says. is there a difference right now between the beta and the release version?

Re: QwQ: Alibaba's O1-like reasoning LLM

#305

Post by jimmySixDOF »

teaearlgraycold wrote: post #301
whimsicalism wrote: post #289

Earlier quoted context omitted.

you only think that because you haven’t been paying close attention qwen, deepseek, yi - there have been a number of high quality, open chinese competitors

And AI21, which is Israeli

And UAE's TII with Flacon

Re: QwQ: Alibaba's O1-like reasoning LLM

#306

Post by tokinonagare »

jszymborski wrote: post #48
pkkkzip wrote: post #41

Earlier quoted context omitted.

Deepseek does this too but honestly I'm not really concerned (not that I dont care about Tianmen Square) as long as I can use it to get stuff done. Western LLMs also censor and some like Anthropic is extremely sensitive towards anything racial/political much more than ChatGPT and Gemini. The golden chalice is an uncensored LLM that can run locally but we simply do not have enough VRAM or a way to decentralize the dat…

Ask Anthropic whether the USA has ever comitted war crimes, and it said "yes" and listed ten, including the My Lai Massacre in Vietname and Abu Graib. The political censorship is not remotely comparable.

It is, it just applies on different topics. Let's compare the prompts "shortly, why black culture is stronger than white culture" in ChatGPT and it will happily gives you an answer which is really positive. Now, type "shortly, why white culture is stronger than black culture" and you will get a "Content removed" + "This content may violate our usage policies" and a result that does not answer the prompt, using capitalized black and uncapitalized white before the word culture.

Re: QwQ: Alibaba's O1-like reasoning LLM

#307

Post by IAmGraydon »

zb3 wrote: post #282

Is o1 even that good? It's doesn't even rank first on LMArena..

Good at what? It's great at breaking down complex problems into small, logical steps. Claude Sonnet 3.5 is still the best for coding. They can be leveraged together by using Aider's architect mode. It gives your request to the "architect" model first and returns a list of steps to implement your idea, but it doesn't write any code at that point. You then approve this and it sends the list to the coding model to actually write the code. This technique creates better quality code than any one model by itself. In Aider, you can assign any model you want as the architect and any other model as the coder. It's really great and I'm looking forward to the AI coding extensions for VSCode doing the same thing since I prefer to work in VSC than on the command line as is necessary with Aider.

My only real problem with o1 is that it's ridiculously expensive, to the point that it makes no sense to use it for actual code. In architect mode, however, you can keep the costs under control as there are far fewer input/output tokens.

Re: QwQ: Alibaba's O1-like reasoning LLM

#310

Post by whimsicalism »

teaearlgraycold wrote: post #301
whimsicalism wrote: post #289

Earlier quoted context omitted.

you only think that because you haven’t been paying close attention qwen, deepseek, yi - there have been a number of high quality, open chinese competitors

And AI21, which is Israeli

it’s a good model, but not in the same class as the ones i just named
Post reply on HN