Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

twitter.com

Post reply on HN
Print view View original

41–50 of 194 posts

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#41

Post by hereme888 »

homarp wrote: post #32
refulgentis wrote: post #27

Earlier quoted context omitted.

The amount of people who will be using it at 1 token/sec because there's no better option, and have 64 GB of RAM, is vanishingly small. IMHO it sets the local LLM community back when we lean on extreme quantization & streaming weights from disk to say something is possible*, because when people try it out, it turns out it's an awful experience. * the implication being, anything is possible in that scenario

agentic loop can run all night long. It's just a different way to work: prepare your prompt queue, set it up, check result in the morning, adjust. 'local vibe' in 10h instead of 10mn is still better than 10 days of manual side coding.

Right on! Especially if its coding abilities are better than Claude 4 Opus. I spent thousands on my PC in anticipation of this rather than to play fancy video games.

Now, where's that spare SSD...

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#42

Post by viraptor »

How well separated are experts per domain in a model like that? Specifically, if I'm interested in a programming use only, could we possibly strip it to one or two of them? Or should I assume a much wider spread? (And there would be some overlap anyway from the original root model)

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#43

Post by orbital-decay »

viraptor wrote: post #42

How well separated are experts per domain in a model like that? Specifically, if I'm interested in a programming use only, could we possibly strip it to one or two of them? Or should I assume a much wider spread? (And there would be some overlap anyway from the original root model)

Inseparable, routing is done per token in a statistically optimal way, not per request on the knowledge domain basis.

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#44

Post by renonce »

viraptor wrote: post #42

How well separated are experts per domain in a model like that? Specifically, if I'm interested in a programming use only, could we possibly strip it to one or two of them? Or should I assume a much wider spread? (And there would be some overlap anyway from the original root model)

My experience is that experts are not separated in any intuitive way. I would be very interested (and surprised) if someone manages to prune a majority of experts in a way that preserves model capabilities in a specific domain but not others.

See https://github.com/peteryuqin/Kimi-K2-Mini, a project that keeps a small portion of experts and layers and keep the model capabilities across multiple domains.

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#45

Post by refulgentis »

selfhoster11 wrote: post #38
refulgentis wrote: post #27

Earlier quoted context omitted.

The amount of people who will be using it at 1 token/sec because there's no better option, and have 64 GB of RAM, is vanishingly small. IMHO it sets the local LLM community back when we lean on extreme quantization & streaming weights from disk to say something is possible*, because when people try it out, it turns out it's an awful experience. * the implication being, anything is possible in that scenario

Good. Vanishingly small is still more than zero. Over time, running such models will become easier too, as people slowly upgrade to better hardware. It's not like there aren't options for the compute-constrained either. There are lots of Chinese models in the 3-32B range, and Gemma 3 is particularly good too. I will also point out that having three API-based providers deploying an impractically-large open-weights mod…

You said "Good." then wrote a nice stirring bit about how having a bad experience with a 1T model will force people to try 4B/32B models.

That seems separate from the post it was replying to, about 1T param models.

If it is intended to be a reply, it hand waves about how having a bad experience with it will teach them to buy more expensive hardware.

Is that "Good."?

The post points out that if people are taught they need an expensive computer to get 1 token/second, much less try it and find out it's a horrible experience (let's talk about prefill), it will turn them off against local LLMs unnecessarily.

Is that "Good."?

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#46

Post by [deleted] »

orbital-decay wrote: post #43
viraptor wrote: post #42

How well separated are experts per domain in a model like that? Specifically, if I'm interested in a programming use only, could we possibly strip it to one or two of them? Or should I assume a much wider spread? (And there would be some overlap anyway from the original root model)

Inseparable, routing is done per token in a statistically optimal way, not per request on the knowledge domain basis.

[deleted]

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#47

Post by viraptor »

renonce wrote: post #44
viraptor wrote: post #42

How well separated are experts per domain in a model like that? Specifically, if I'm interested in a programming use only, could we possibly strip it to one or two of them? Or should I assume a much wider spread? (And there would be some overlap anyway from the original root model)

My experience is that experts are not separated in any intuitive way. I would be very interested (and surprised) if someone manages to prune a majority of experts in a way that preserves model capabilities in a specific domain but not others. See https://github.com/peteryuqin/Kimi-K2-Mini , a project that keeps a small portion of experts and layers and keep the model capabilities across multiple domains.

Sounds like dumping the routing information from programming questions would answer that... I guess I can do a dump from qwen or deepseek locally. You'd think someone would created that kind of graph already, but I couldn't find one.

What I did find instead is that some MoE models are explicitly domain-routed (MoDEM), but it doesn't apply to deepseek which is just equally load balanced, so it's unlikely to apply to Kimi. On the other hand, https://arxiv.org/html/2505.21079v1 shows modality preferences between experts, even in mostly random training. So maybe there's something there.

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#48

Post by viraptor »

orbital-decay wrote: post #43
viraptor wrote: post #42

How well separated are experts per domain in a model like that? Specifically, if I'm interested in a programming use only, could we possibly strip it to one or two of them? Or should I assume a much wider spread? (And there would be some overlap anyway from the original root model)

Inseparable, routing is done per token in a statistically optimal way, not per request on the knowledge domain basis.

Sure, it's done per token, but the question is: how much do the knowledge domains match up with experts. I could not find hard data on this.

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#49

Post by sergiotapia »

qmmmur wrote: post #39
simonw wrote: post #17

Pelican on a bicycle result: https://simonwillison.net/2025/Jul/11/kimi-k2/

I'm glad we are looking to build nuclear reactors so we can do more of this...

me too - we must energymaxx. i want a nuclear reactor in my backyard powering everything. I want ac units in every room and my open door garage while i workout.
Post reply on HN