Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

twitter.com

Post reply on HN
Print view View original

121–130 of 194 posts

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#121

Post by overfeed »

fzzzy wrote: post #97
segmondy wrote: post #92

Earlier quoted context omitted.

I do deepseek at 5tk/sec at home and I'm happy with it. I don't need to do agent stuff to gain from it, I was saving to eventually build out enough to run it at 10tk/sec, but with kimi k2, plan has changed and the savings continue with a goal to run it at 5 tk/sec at home.

I agree, 5 tokens per second is plenty fast for casual use.

Also works perfectly fine in fire-and-forget, non-interactive agentic workflows. My dream scenario is that I create a bunch of kanban tickets and assign them to one or more AI personas[1], and wake up to some Pull Requests the next morning. I'd me more concerned about tickets-per-day, and not tk/s as I have no interest in watching the inner-workings of the model.

1. Some more creative than others, with slightly different injected prompts or perhaps even different models entirely.

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#122

Post by [deleted] »

pxc wrote: post #119
johnb231 wrote: post #108

Earlier quoted context omitted.

These are all "too big to host at home". I don't think that is the issue here. https://github.com/MoonshotAI/Kimi-K2/blob/main/docs/deploy_... "The smallest deployment unit for Kimi-K2 FP8 weights with 128k seqlen on mainstream H200 or H20 platform is a cluster with 16 GPUs with either Tensor Parallel (TP) or "data parallel + expert parallel" (DP+EP)." 16 GPUs costing ~$30k each. No one is running a ~$500k server at…

I think what GP means is that because the (hopefully) pending OpenAI release is also "too big to run at home", these two models may be close enough in size that they seem more directly comparable, meaning that it's even more important for OpenAI to outperform Kimi K2 on some key benchmarks.

[deleted]

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#123

Post by ls612 »

johnb231 wrote: post #108
irthomasthomas wrote: post #72

Earlier quoted context omitted.

Someone at openai did say it was too big to host at home, so you could be right. They will probably be benchmaxxing, right now, searching for a few evals they can beat.

These are all "too big to host at home". I don't think that is the issue here. https://github.com/MoonshotAI/Kimi-K2/blob/main/docs/deploy_... "The smallest deployment unit for Kimi-K2 FP8 weights with 128k seqlen on mainstream H200 or H20 platform is a cluster with 16 GPUs with either Tensor Parallel (TP) or "data parallel + expert parallel" (DP+EP)." 16 GPUs costing ~$30k each. No one is running a ~$500k server at…

This is a dumb question I know, but how expensive is model distillation? How much training hardware do you need to take something like this and create a 7B and 12B version for consumer hardware?

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#124

Post by tonyhart7 »

Alifatisk wrote: post #111
alt187 wrote: post #107

Earlier quoted context omitted.

What part of this goes against the four fundamental freedoms? Can you point at it?

Exactly, I wouldn’t mind adding that text on our service if we made 20m $, the parent made it sound like a huge clause

Yeah, its fair for them if they want a little bit credit

nothing gucci there

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#125

Post by Intermernet »

echelon wrote: post #83
satvikpendem wrote: post #75

This is not open source, they have a "modified MIT license" where they have other restrictions on users over a certain threshold. Our only modification part is that, if the Software (or any derivative works thereof) is used for any of your commercial products or services that have more than 100 million monthly active users, or more than 20 million US dollars (or equivalent in other currencies) in monthly revenue, you…

> This is not open source OSI purism is deleterious and has led to industry capture. Non-viral open source is simply a license for hyperscalers to take advantage. To co-opt offerings and make hundreds of millions without giving anything back. We need more "fair source" licensing to support sustainable engineering that rewards the small ICs rather than mega conglomerate corporations with multi-trillion dollar market c…

Yep, awesome stuff. Call it "fair source" if you want to. Don't call it open source. I'm an absolutist about very few things, but the definition of open source is one of them. Every bit of variation given in the definition is a win for those who have ulterior motives for polluting the definition. Open source isn't a vague concept, it's a defined term with a legally accepted meaning. Very much like "fair use". It's dangerous to allow this definition to be altered. OpenAI (A deliberate misnomer if ever there was one) and friends would really love to co-opt the term.

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#126

Post by a2128 »

simonw wrote: post #115
alt187 wrote: post #107

Earlier quoted context omitted.

What part of this goes against the four fundamental freedoms? Can you point at it?

"The freedom to run the program as you wish, for any purpose (freedom 0)." Being required to display branding in that way contradicts "run the program as you wish".

Being required to store the GPL license notice on my hard drive is contradicting my wishes. And I'm not even earning $20 million US dollars per month off GPL software!

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#127

Post by wongarsu »

mindcrime wrote: post #104
kragen wrote: post #78

Earlier quoted context omitted.

I feel like those restrictions don't violate the OSD (or the FSF's Free Software Definition, or Debian's); there are similar restrictions in the GPLv2, the GPLv3, the 4-clause BSD license, and so on. They just don't have user or revenue thresholds. The GPLv2, for example, says: > c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use…

It may not violate the OSD, but I would still argue that this license is a Bad Idea. Not because what they're trying to do is inherently bad in any way, but simply because it's yet another new, unknown, not-fully-understood license to deal with. The fact that we're having this conversation illustrating that very fact. My personal feeling is that almost every project (I'll hedge a little because life is complicated) s…

Aren't most licenses "not fully understood" in any reasonable legal sense? To my knowledge only the Artistic License and the GPL have seen the inside of a court room. And yet to this day nobody really knows how the GPL works with languages that don't follow C's model of a compile and a link step. And the boundaries of what's a derivative work in the GPL are still mostly set by convention, not a legal framework.

What makes us comfortable with the "traditional open source licenses" is that people have been using them for decades and nothing bad has happened. But that's mostly because breaking an open source license is rarely litigated against, not because we have some special knowledge of what those licenses mean and how to abide by that

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#128

Post by kachapopopow »

gpm wrote: post #93
summarity wrote: post #64

Earlier quoted context omitted.

Reasonable speeds are possible with 4bit quants on 2 512GB Mac Studios (MLX TB4 Ring - see https://x.com/awnihannun/status/1943723599971443134 ) or even a single socket Epyc system with >1TB of RAM (about the same real world memory throughput as the M Ultra). So $20k-ish to play with it. For real-world speeds though yeah, you'd need serious hardware. This is more of a "deploy your own stamp" model, less a "local" mod…

> or even a single socket Epyc system with >1TB of RAM How many tokens/second would this likely achieve?

around 1 by the time you try to do anything useful with it (>10000 tokens)

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#129

Post by weitendorf »

simonw wrote: post #115
alt187 wrote: post #107

Earlier quoted context omitted.

What part of this goes against the four fundamental freedoms? Can you point at it?

"The freedom to run the program as you wish, for any purpose (freedom 0)." Being required to display branding in that way contradicts "run the program as you wish".

You are still free to run the program as you wish, you just have to provide attribution to the end user. It's essentially CC BY but even more permissive, because the attribution only kicks in once when specific, relatively uncommon conditions are met.

I think basically everybody considers CC BY to be open source, so a strictly more permissive license should be too, I think.

Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model

#130

Post by wongarsu »

summarity wrote: post #64
vessenes wrote: post #62

I tried Kimi on a few coding problems that Claude was spinning on. It’s good. It’s huge, way too big to be a “local” model — I think you need something like 16 H200s to run it - but it has a slightly different vibe than some of the other models. I liked it. It would definitely be useful in ensemble use cases at the very least.

Reasonable speeds are possible with 4bit quants on 2 512GB Mac Studios (MLX TB4 Ring - see https://x.com/awnihannun/status/1943723599971443134 ) or even a single socket Epyc system with >1TB of RAM (about the same real world memory throughput as the M Ultra). So $20k-ish to play with it. For real-world speeds though yeah, you'd need serious hardware. This is more of a "deploy your own stamp" model, less a "local" mod…

Reasonable speeds are possible if you pay someone else to run it. Right now both NovitaAI and Parasail are running it, both available through Openrouter and both promising not to store any data. I'm sure the other big model hosters will follow if there's demand.

I may not be able to reasonably run it myself, but at least I can choose who I trust to run it and can have inference pricing determined by a competitive market. According to their benchmarks the model is about in a class with Claude 4 Sonet, yet already costs less than one third of Sonet's inference pricing

Post reply on HN