Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi-K3 on HuggingFace

huggingface.co

Post reply on HN
Print view View original

451–460 of 588 posts

Re: Kimi-K3 on HuggingFace

#451

Post by fnordpiglet »

I suggest downloading these frontier models just to have a copy; even though it’s 1.5TB, it’s worth sticking in a cheap disk and putting aside. Seeding torrents would be even more useful. The man is coming to lock these down, like they tried to do with encryption algorithms. The only way open software survives regulation is through distribution.

Over time the enormous investment in techniques and hardware manufacturing will almost certainly make these runnable in a more practical way. It will be a shame if by the time we get there it’s illegal to distribute them and you have to pay a reg capture premium and feed the machine.

Re: Kimi-K3 on HuggingFace

#453

Post by withinboredom »

johndough wrote: post #416
withinboredom wrote: post #274

Earlier quoted context omitted.

704gb -> 564gb; 358 gb -> 270 gb; 28.79 gb -> 7.65 gb; 439 gb -> 93 gb It depends on the total entropy of the model. Smaller models have less entropy.

> Smaller models have less entropy. Interesting. Why is that? I would have expected the opposite, since larger models have to try less hard to fit the training data. Or maybe this leaves more parameters with random initialization, resulting in higher entropy for larger models?

I honestly don't know... I didn't train the models, so I can't tell why the math works out that way. It just does. I suspect it has to do with the fact that all the small models I've tested have been quantized. I don't know of any small model trained from scratch. If you know of any, I'd be happy to encode it and see what it looks like.

Re: Kimi-K3 on HuggingFace

#455

Post by jmalicki »

walrus01 wrote: post #417
tiahura wrote: post #403

Earlier quoted context omitted.

There are a number of use cases where sending the contents of your context and prompts (and the resulting output) to a 3rd party service is off the table as an option, and people will compromise speed for data sovereignty. Are there? At the highest levels of defense and law, AWS and Azure are used. Having tried selling some of these entities on doing things in-house, there seems to be little interest.

> Are there? At the highest levels of defense and law, AWS and Azure are used. This is certainly true if the user is an American company. You could look at the European initiatives to run this stuff on hardware they own in facilities they own and control within the borders of Europe for a counter-example. Such as: https://www.google.com/search?client=firefox-b-d&q=schwarz+s... https://www.dutchnews.nl/2026/04/governm…

Yeah, true European cloud providers for these kinds of things seem to be behind, and a lot of the ones offering data compliance at the level of AWS are small enough that it's a bit harder to trust they'll be around and will keep their promises.

Hopefully that changes!

Re: Kimi-K3 on HuggingFace

#456

Post by nancyminusone »

fnordpiglet wrote: post #451

I suggest downloading these frontier models just to have a copy; even though it’s 1.5TB, it’s worth sticking in a cheap disk and putting aside. Seeding torrents would be even more useful. The man is coming to lock these down, like they tried to do with encryption algorithms. The only way open software survives regulation is through distribution. Over time the enormous investment in techniques and hardware manufacturi…

they will just restrict you from buying the hardware these run on

Re: Kimi-K3 on HuggingFace

#457

Post by mvdtnz »

fnordpiglet wrote: post #451

I suggest downloading these frontier models just to have a copy; even though it’s 1.5TB, it’s worth sticking in a cheap disk and putting aside. Seeding torrents would be even more useful. The man is coming to lock these down, like they tried to do with encryption algorithms. The only way open software survives regulation is through distribution. Over time the enormous investment in techniques and hardware manufacturi…

There's no such thing as cheap disk anymore and I'm not spending hundreds of dollars out of paranoia of "the man".

Re: Kimi-K3 on HuggingFace

#458

Post by lightningdev1 »

minimaltom wrote: post #452

HF says activation function is "SiTU-GLU", but I can't find any info on that? Anyone know / is this a typo? Even the python code for inference seems to use normal activations.

It stands for Sigmoid Tanh Unit Gated Linear Unit. Check the tech report/paper (page 7): https://github.com/MoonshotAI/Kimi-K3/blob/main/k3_tech_repo...

Re: Kimi-K3 on HuggingFace

#459

Post by withinboredom »

withinboredom wrote: post #288
xscott wrote: post #173

Earlier quoted context omitted.

Exploring compression algorithms for weights is a good idea, and I hope you have a successful product. However, if you can prove this statement: > reduces it down to its minimum entropy -- it cannot be compressed further. I think you could make a lot more money elsewhere :-) https://en.wikipedia.org/wiki/Kolmogorov_complexity#Formal_p...

We're not an AI company ... nor do we have any reason to use it. Just a fun idea that was fruitful.

And to clarify -- it only applies to models, not arbitrary data. So its useful to exactly zero other fields.

Re: Kimi-K3 on HuggingFace

#460

Post by bjackman »

Tepix wrote: post #214
bjackman wrote: post #104

Earlier quoted context omitted.

I agree but worth noting that it's never gonna be very practical to run LLMs like this at home. Unless we have some sort of design breakthrough, the only "sensible" way to run them is at high batch levels on shared HW. Like, yeah if I could spend a few grand on such a GPU I probably would coz I'm a rich nerd, but I'd acknowledge it as an extremely inefficient luxury, kinda like a sports car. So I think you could say…

"Never" is a long time. Just think about how much ram we had 10 or 20 years ago. 1.5TB isn't a lot really.

It doesn't matter if you have the RAM, running a 1.5TB model for a single context stream is fundamentally inefficient.
Post reply on HN