191–200 of 435 posts
Earlier quoted context omitted.
What hardware are you able to run this on?
I am running it on a 32G memory mac mini with an M2 Pro using Ollama. It runs fine, faster than I expected. The way it explains plans for solving problems, then proceeding step by step is impressive.
This one is pretty impressive. I'm running it on my Mac via Ollama - only a 20GB download, tokens spit out pretty fast and my initial prompts have shown some good results. Notes here: https://simonwillison.net/2024/Nov/27/qwq/
What hardware are you able to run this on?
See also Hugging Face's MLX community:
https://huggingface.co/mlx-community
QwQ 32B is featured:
https://huggingface.co/collections/mlx-community/qwq-32b-pre...
If you want a traditional GUI, LM Studio beta 0.3.x is iterating on MLX: https://lmstudio.ai/beta-releases
Earlier quoted context omitted.
Forgot about R1, what hardware are you using to run it?
I haven’t ran QWQ yet, but it’s a 32B. So about 20GB RAM with Q4 quant. Closer to 25GB for the 4_K_M one. You can wait for a day or so for the quantized GGUFs to show up (we should see the Q4 in the next hour or so). I personally use Ollama on an MacBook Pro. It usually takes a day or two for it to show up. Any M series MacBook with 32GB+ of RAM will run this.
https://huggingface.co/collections/mlx-community/qwq-32b-pre...
For a GUI, LM Studio 0.3.x is iterating MLX support: https://lmstudio.ai/beta-releases
When searching in LM Studio, you can narrow search to the mlx-community.
Earlier quoted context omitted.
"If you're China" clearly refers to the government/party, assuming otherwise isn't good faith.
When you say this, I don't think any Chinese people actually believe you.
This will also hold for whether they believe us - in that too, Chinese people won't be a monolith. Plenty of those who aren't the biggest fans of the CCP will, as they understand where we're coming from better than anyone.
Earlier quoted context omitted.
Ask Anthropic whether the USA has ever comitted war crimes, and it said "yes" and listed ten, including the My Lai Massacre in Vietname and Abu Graib. The political censorship is not remotely comparable.
> The political censorship is not remotely comparable. Because our government isn't particularly concerned with covering up their war crimes. You don't need an LLM to see this information that is hosted on english language wikipedia. American political censorship is fought through culture wars and dubious claims of bias.
What you are describing are social moires and norms. It is not related to political censorship by the government.
Earlier quoted context omitted.
> the spirit of the sanctions What does this mean? The sanctions are very specific on what can't be sold, so the spirit is to sell anything up to that limit.
> What does this mean? The sanctions are very specific on what can't be sold, so the spirit is to sell anything up to that limit. 25% of Nvidia revenue comes from the tiny country of Singapore. You think Nvidia is asking why? (Answer: they aren’t)
Earlier quoted context omitted.
Because training usually requires bigger batches, doing a backward pass instead of just the forward pass, storing optimizer states in memory etc. This means it takes a lot more RAM than inference, so much more that you can't run it on a single GPU. If you're training on more than one GPU, the speed at which you can exchange data between them suddenly becomes your bottleneck. To alleviate that problem, you need extrem…
But does this prevent usage of cluster or consumer GPUs to be used in training? Or does it just make it slower and less efficient? Those are real questions and not argumentative questions.
Earlier quoted context omitted.
From the link > Performance and Benchmark Limitations: The model excels in math and coding but has room for improvement in other areas, such as common sense reasoning and nuanced language understanding.
Oh, so they made an autistic LLM
Earlier quoted context omitted.
That is a pretty dark view on almost 1/5th of humanity and a nation with a track record of giving the world important innovations: paper making, silk, porcelain, gunpowder and compass to name the few. Not everything has to be around politics.
Also a nation that just used their cargo ship to deliberately cut two undersea cables. But I guess that's not about politics either?
Earlier quoted context omitted.
I don’t see why they wouldn’t. If you’re China and willing to pour state resources into LLMs, it’s an incredible ROI if they’re adopted. LLMs are black boxes, can be fine tuned to subtly bias responses, censor, or rewrite history. They’re a propaganda dream. No code to point to of obvious interference.
That is a pretty dark view on almost 1/5th of humanity and a nation with a track record of giving the world important innovations: paper making, silk, porcelain, gunpowder and compass to name the few. Not everything has to be around politics.