[dead]
Kimi-K3 on HuggingFace
171–180 of 588 posts
Re: Kimi-K3 on HuggingFace
#172Did someone run censorship and political bias tests on this ? Must be interesting.
It would, indeed, be interesting to compare, given what we know about Anthropic’s censorship and political bias in their closed and more expensive models.
https://x.com/dhh/status/2081435006770249831 (from the creator of Ruby on Rails).
In his specific case Kimi did the task it was asked to do (translation of the article DHH wrote), which Claude refused to.
Re: Kimi-K3 on HuggingFace
#173This will be interesting for a few reasons. First, depending on where the median pricing settles w/ 3rd party providers will tell us what it costs to serve a 3T model. Since it's going to be mxfp4 native, it'll take ~1.5TB of VRAM to host this, which is juuust at the limit of 8xb200s (but realistically you'll need 16x for context / throughput optimisation). Won't be cheap to host, but at least we should get some rang…
We have a lossless compression codec (working on open sourcing it over the next couple of weeks) that reduces it down to its minimum entropy -- it cannot be compressed further. On all tested large models, it's a ratio of 1.34-1.23 -- and smaller models up to 3.76x. It also increases the effective bandwidth by the same rate.
Exploring compression algorithms for weights is a good idea, and I hope you have a successful product. However, if you can prove this statement:
> reduces it down to its minimum entropy -- it cannot be compressed further.
I think you could make a lot more money elsewhere :-)
https://en.wikipedia.org/wiki/Kolmogorov_complexity#Formal_p...
Re: Kimi-K3 on HuggingFace
#174Re: Kimi-K3 on HuggingFace
#175I heard this is the talk in town these days. Why can't Meta keep up? With >10000000x more resources you'd think that they'd be able to introduce equally performant if not better open weight models
At this point in time what does meta get from releasing open weight models? Why devote the resources to it.
Why would there be any motivated person left in this place.
Re: Kimi-K3 on HuggingFace
#176This has to be one of the craziest uploads on the internet up until now.
Raw fucking intelligence at your disposal, free to download.
If you'd describe what's happening now to someone from five years ago they'd think you're hallucinating or mad.
Re: Kimi-K3 on HuggingFace
#177I feel like most hardware to run LLMs on is shaped wrong for individuals. It's either having a model struggling along with like 5-10 tokens per second on unified memory, or data center cards with hundreds of GB of VRAM consuming more than a kW of power. It doesn't seem like there's prosumer GPUs with like 180W-250W TDP and 128 GB or 256 GB of VRAM (one can dream). Then bifurcation and even just two of those cards wou…
I've been feeling for a while that as we keep increasing model size, we're going through the opposite of the PC revolution.
The "democratisation" talk from the frontier labs is especially egregious when they only release closed models (gpt-oss hardly counts) and are trying everything they can to make it harder to run open models.
Re: Kimi-K3 on HuggingFace
#178FYI huggingface refers to the alien from the Aliens movies that we need to prevent from reaching earth at any cost because it means the end of civilization. Just checking in because y'all sound good with that.
haha stupid xenomorphs with their acid blood and pointy bits - all they had to do is make the beasts write code and do our homework :)
Re: Kimi-K3 on HuggingFace
#179Earlier quoted context omitted.
The individual-shaped-hardware problem gets even sharper at the phone end. Shipping a 3B model on-device, the usable RAM budget after the OS and everything else is more like 2-4GBtotal, not per-model so it's not 'can I afford more VRAM', it's 'can I fit a language model and an STT model and embeddings without the OS killing my process'. Feels like phones are the most hardware-constrained 'individual' tier and get the…
Recently spent a few hours messing with bonsai 27B and ternary bonsai 27B at total weights + context fitting in slightly under 6GB RAM, and it's just dumb as hell. It writes what seems like grammatically correct content but it's extremely limited. It will also happily hallucinate new names and content to fill in gaps in its knowledge, and present the hallucations in what looks like a correctly formatted sentence, so…
[dead]
Re: Kimi-K3 on HuggingFace
#180FYI huggingface refers to the alien from the Aliens movies that we need to prevent from reaching earth at any cost because it means the end of civilization. Just checking in because y'all sound good with that.
Doesn’t it refer to the emoji?