Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

twitter.com

Post reply on HN
Print view View original

31–40 of 194 posts

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#32

Post by homarp »

refulgentis wrote: post #27
selfhoster11 wrote: post #24

Earlier quoted context omitted.

It does not have to be VRAM, it could be system RAM, or weights streamed from SSD storage. Reportedly, the latter method achieves around 1 token per second on computers with 64 GB of system RAM. R1 (and K2) is MoE, whereas Llama 3 is a dense model family. MoE actually makes these models practical to run on cheaper hardware. DeepSeek R1 is more comfortable for me than Llama 3 70B for exactly that reason - if it spills…

The amount of people who will be using it at 1 token/sec because there's no better option, and have 64 GB of RAM, is vanishingly small. IMHO it sets the local LLM community back when we lean on extreme quantization & streaming weights from disk to say something is possible*, because when people try it out, it turns out it's an awful experience. * the implication being, anything is possible in that scenario

agentic loop can run all night long. It's just a different way to work: prepare your prompt queue, set it up, check result in the morning, adjust. 'local vibe' in 10h instead of 10mn is still better than 10 days of manual side coding.

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#33

Post by simonw »

ebiester wrote: post #31
simonw wrote: post #17

Pelican on a bicycle result: https://simonwillison.net/2025/Jul/11/kimi-k2/

At this point, they have to be training it. At what point will you start using something else?

Once I get a picture that genuinely looks like a pelican riding a bicycle!

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#35

Post by adt »

gs17 wrote: post #2

> 1T total / 32B active MoE model Is this the largest open-weight model?

No.

At 1T MoE on 15.5T tokens, K2 is one of the largest open source models to date. But BAAI's TeleFM is 1T dense on 15.7T tokens: https://huggingface.co/CofeAI/Tele-FLM-1T

You can always check here: https://lifearchitect.ai/models-table/

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#38

Post by selfhoster11 »

refulgentis wrote: post #27
selfhoster11 wrote: post #24

Earlier quoted context omitted.

It does not have to be VRAM, it could be system RAM, or weights streamed from SSD storage. Reportedly, the latter method achieves around 1 token per second on computers with 64 GB of system RAM. R1 (and K2) is MoE, whereas Llama 3 is a dense model family. MoE actually makes these models practical to run on cheaper hardware. DeepSeek R1 is more comfortable for me than Llama 3 70B for exactly that reason - if it spills…

The amount of people who will be using it at 1 token/sec because there's no better option, and have 64 GB of RAM, is vanishingly small. IMHO it sets the local LLM community back when we lean on extreme quantization & streaming weights from disk to say something is possible*, because when people try it out, it turns out it's an awful experience. * the implication being, anything is possible in that scenario

Good. Vanishingly small is still more than zero. Over time, running such models will become easier too, as people slowly upgrade to better hardware. It's not like there aren't options for the compute-constrained either. There are lots of Chinese models in the 3-32B range, and Gemma 3 is particularly good too.

I will also point out that having three API-based providers deploying an impractically-large open-weights model beats the pants of having just one. Back in the day, this was called second-sourcing IIRC. With proprietary models, you're at the mercy of one corporation and their Kafkaesque ToS enforcement.

Post reply on HN