Did someone run censorship and political bias tests on this ? Must be interesting.
There's is always one in each and every AI forum: "But, but have you asked it about tiananmen"?
Kimi-K3 on HuggingFace
311–320 of 588 posts
Re: Kimi-K3 on HuggingFace
#312Earlier quoted context omitted.
On the flip side, I really like being able to inspect its reasoning chain thoroughly, as opposed to the "black box" that Anthropic models are now.
Right I was going to say, no way of knowing whether these issues are unique to Chinese models.
Depends on whether the models report the correct amount of tokens.
5.5 Sol repors 10x fewer reasoning tokens than Kimi k3. If it is correct, than it unlikely has those doubt issues.
At the same time, I feel like their reporting is incorect and we are now paying per "intelligence", not actual tokens. We can't verify it anyway..
Re: Kimi-K3 on HuggingFace
#313There's no going back on this. This is putting a very capable intelligence in the hands of the masses. Private companies in the US are aching for Trump's protectionism but it'll do nothing. The hardware needed to run this is ofc prohibitive, but actually putting it out there feels like a 'RSA source code on t-shirt' moment for humanity.
I don't know if the masses can quite afford the 500k in GPUs you need to run this
Re: Kimi-K3 on HuggingFace
#314This will be interesting for a few reasons. First, depending on where the median pricing settles w/ 3rd party providers will tell us what it costs to serve a 3T model. Since it's going to be mxfp4 native, it'll take ~1.5TB of VRAM to host this, which is juuust at the limit of 8xb200s (but realistically you'll need 16x for context / throughput optimisation). Won't be cheap to host, but at least we should get some rang…
[flagged]
Re: Kimi-K3 on HuggingFace
#315Earlier quoted context omitted.
We know labs make money on inference, and we know they lose a lot of money on inference+training.
> We know labs make money on inference We don't really know that, for OpenAI and Anthropic. We suspect that, but as far as I know, even they have stopped claiming that they are profitable on inference.
unless you think that Opus is 10T+ params, its pretty much impossible for inference not to be profitable when doing some basic napkin math on other open models, and if Kimi K3 is 3T params with the same performance as Opus then that means that China is actually way more technologically advanced than the American labs.
So which is it?
Re: Kimi-K3 on HuggingFace
#316Wait, so I can download it and run it locally now?? Wow... But it probably won't work on my computer, right?
The short answer is no, it won't work on your home computer. In it's current form it needs something like 594 GB of memory, far outside what you can reasonably run on normal consumer hardware in 2026. If you have really high end hardware, you might be able to squeeze a heavily quantized version of Kimi-K3 onto your rig, but it will be too slow or too lobotomized to be useful. This does put a near state-of-the-art ope…
[dead]
Re: Kimi-K3 on HuggingFace
#317This is historic. For the first time, an open-weights LLM is right at the top. We won't be able to run this ourselves, but many providers can.
> For the first time, an open-weights LLM is right at the top. Hmm, not quite true, I think that honor, for better or worse, goes to OpenAI. When they released GPT2 (or GPT1 for that matter) is was quite literally the SOTA in the ecosystem when it was released.
You are correct. I miss the time when OpenAI was open.
Re: Kimi-K3 on HuggingFace
#318This will be interesting for a few reasons. First, depending on where the median pricing settles w/ 3rd party providers will tell us what it costs to serve a 3T model. Since it's going to be mxfp4 native, it'll take ~1.5TB of VRAM to host this, which is juuust at the limit of 8xb200s (but realistically you'll need 16x for context / throughput optimisation). Won't be cheap to host, but at least we should get some rang…
pardon my ignorance but is fine tuning still considered viable in face of rapid model releases. Is it really worth it?
my friends whove tried in their companies gave up on it.
Re: Kimi-K3 on HuggingFace
#319K3 releasing as open source right at the same time that people are criticizing Opus for questionable performance (there's even a thread on HN about Opus 5's problems)... this is such a flex.
> (there's even a thread on HN about Opus 5's problems
is opus 5 a flop like 4.8 ?
where is the thread btw curios
Re: Kimi-K3 on HuggingFace
#320I feel like most hardware to run LLMs on is shaped wrong for individuals. It's either having a model struggling along with like 5-10 tokens per second on unified memory, or data center cards with hundreds of GB of VRAM consuming more than a kW of power. It doesn't seem like there's prosumer GPUs with like 180W-250W TDP and 128 GB or 256 GB of VRAM (one can dream). Then bifurcation and even just two of those cards wou…
The individual-shaped-hardware problem gets even sharper at the phone end. Shipping a 3B model on-device, the usable RAM budget after the OS and everything else is more like 2-4GBtotal, not per-model so it's not 'can I afford more VRAM', it's 'can I fit a language model and an STT model and embeddings without the OS killing my process'. Feels like phones are the most hardware-constrained 'individual' tier and get the…
[deleted]