Is their repo / model free of any undisclosed telemetry, ie is it purely weights
These days "open" models come in standard formats. And they are mostly just data.
QwQ: Alibaba's O1-like reasoning LLM
321–330 of 435 posts
Re: QwQ: Alibaba's O1-like reasoning LLM
#322So western controls on training hardware (hello NVIDIA) seem to have failed. I wonder if there will be any repercussions here.
Most of the papers in machine learning are coming from China. The vast majority. Most of the open source models on GitHub, too. Hailuo, Kling, Vidu, and Hunyuan (posted on Banodoko) blow Sora and Runway out of the water. China is dominating at this field. And if they begin releasing weights as open source, that'll mean foundation model companies can only bank on the thin facade of product. That's a really good strate…
I don't know enough to comment but can someone downvoting this please explain why they are doing so? Is it untrue, an exaggeration, something else or maybe simply an observation they don't like to see.
Re: QwQ: Alibaba's O1-like reasoning LLM
#323This one is crazy. I made up a silly topology problem which I guessed wouldn't be in a textbook (given X create a shape with Euler characteristic X) and set it to work. Its first effort was a program that randomly generated shapes, calculated X and hoped it was right. I went and figured out a solution and gave it a clue. Watching it "think" through the answer is surreal and something I haven't felt since watching GPT…
It's on Wikipedia though ...
Re: QwQ: Alibaba's O1-like reasoning LLM
#324So western controls on training hardware (hello NVIDIA) seem to have failed. I wonder if there will be any repercussions here.
The sanctions only prevent exporting a certain class of GPUs to China. There's nothing stopping Chinese companies from setting up their operations overseas (ahem Singapore), or even renting space from AWS/Google/Azure.
Re: QwQ: Alibaba's O1-like reasoning LLM
#325So western controls on training hardware (hello NVIDIA) seem to have failed. I wonder if there will be any repercussions here.
Most of the papers in machine learning are coming from China. The vast majority. Most of the open source models on GitHub, too. Hailuo, Kling, Vidu, and Hunyuan (posted on Banodoko) blow Sora and Runway out of the water. China is dominating at this field. And if they begin releasing weights as open source, that'll mean foundation model companies can only bank on the thin facade of product. That's a really good strate…
> That's a really good strategy to make sure American AI startups don't achieve escape velocity if they have to fend of dozens of fungible clones.
A strategy pioneered by Meta.
Re: QwQ: Alibaba's O1-like reasoning LLM
#326Earlier quoted context omitted.
Or they could be training the models in the states? It’s hard to say since alibaba does R&D in Bellevue as well as Hangzhou.
My understanding is that R&D in the US is a very small part of the business for these China-based companies, both in terms of headcounts and org chart. TikTok hires many people in the US, but that's nowhere near the number in the Beijing office, not to mention that all decisions come from China. You can find lots of articles that talk about midnight meetings for folks in US. And there is a good reason for that -- you…
I visited their office in Bellevue once and it wasn’t small, although not huge either. There were a lot of smart people working there at least. You don’t really need hardware in China to train models, you could host it in the USA and just use it remotely. I don’t think these sanctions will be very effective unless they completely cut off trade and communication with China, which is going to hurt us as much as them.
Re: QwQ: Alibaba's O1-like reasoning LLM
#327Earlier quoted context omitted.
you only think that because you haven’t been paying close attention qwen, deepseek, yi - there have been a number of high quality, open chinese competitors
Thanks. You're right, I am a layman. I may also have been focusing on the "open" LLMs since they seem to get the most talked about on HN.
What do you mean by "open"? AFAIK Qwen, Deepseek etc are just as open as the rest.
Re: QwQ: Alibaba's O1-like reasoning LLM
#328I am right now playing with it running it locally using ollama. It is a 19GB download and it runs nicely on a nvidia A100 GPU. https://ollama.com/library/qwq
Runs nicely on my AMD 7900XTX too.
How are AMD cards performing? I heard it was still very hit and miss in regards to support. Have you also tried things like F5-TTS?
Re: QwQ: Alibaba's O1-like reasoning LLM
#32932B is a good choice of size, as it allows running on a 24GB consumer card at ~4 bpw (RTX 3090/4090) while using most of the VRAM. Unlike llama 3.1, which had 8b, 70B (much too big to fit), and 405B.
what do you mean? I can easily run 70b on my macbook. Fits easily.
Re: QwQ: Alibaba's O1-like reasoning LLM
#33032B is a good choice of size, as it allows running on a 24GB consumer card at ~4 bpw (RTX 3090/4090) while using most of the VRAM. Unlike llama 3.1, which had 8b, 70B (much too big to fit), and 405B.
what do you mean? I can easily run 70b on my macbook. Fits easily.
Macs that can run it are quite a bit more expensive than a 3090. GPUs can also do finetuning and run other models with larger batch sizes which Macs would struggle with. Also, for the models that fit both, an nvidia card can run it much faster.