"Open source" lol Open-weight. As usual, you don't get the dataset, training scripts, etc.
We really need to stop diluting the meaning of open source
Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model
161–170 of 194 posts
Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model
#162Kimi K2 is the large language model series developed by Moonshot AI team.
Moonshot AI [1] (Moonshot; Chinese: 月之暗面; pinyin: Yuè Zhī Ànmiàn) is an artificial intelligence (AI) company based in Beijing, China. As of 2024, it has been dubbed one of China's "AI Tiger" companies by investors with its focus on developing large language models.
I guess everyone is up to date with AI stuff but this is the first time I heard of Kimi and Moonshot and was wondering where it is from. And it wasn't obvious from a quick glance of comments.
Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model
#163Earlier quoted context omitted.
I do deepseek at 5tk/sec at home and I'm happy with it. I don't need to do agent stuff to gain from it, I was saving to eventually build out enough to run it at 10tk/sec, but with kimi k2, plan has changed and the savings continue with a goal to run it at 5 tk/sec at home.
I agree, 5 tokens per second is plenty fast for casual use.
It was, last year 5tk/s was reasonable. If you wanted to proof read a paragraph or rewrite some bullet points into a PowerPoint slide.
Now, with agentic coding, thinking models, a “chat with my pdf” or whatever artifacts are being called now, no, I don’t think 5/s is enough.
Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model
#164This is an open weight model, which is in contrast with closed-source models.
However, 1t parameters makes it nearly impossible for local inference, let alone fine-tuning.
Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model
#165Earlier quoted context omitted.
Reasonable speeds are possible if you pay someone else to run it. Right now both NovitaAI and Parasail are running it, both available through Openrouter and both promising not to store any data. I'm sure the other big model hosters will follow if there's demand. I may not be able to reasonably run it myself, but at least I can choose who I trust to run it and can have inference pricing determined by a competitive mar…
I’m actually finding Claude 4 Sonnet’s thinking model to be too slow to meet my needs. It literally takes several minutes per query on Cursor. So running it locally is the exact opposite of what I’m looking for. Rather, I’m willing to pay more, to have it be run on a faster than normal cloud inference machine. Anthropic is already too slow. Since this model is open source, maybe someone could offer it at a “premium”…
> It literally takes several minutes per query on Cursor.
There's your issue. Use Claude Code or the API directly and compare the speeds. Cursor is slowing down requests to maintain costs.
Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model
#166I love the fact that I can use this right away and test it out in practice. The ecosystem around LLM is simply awesome and improving by the day.
Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model
#167Earlier quoted context omitted.
Good. Vanishingly small is still more than zero. Over time, running such models will become easier too, as people slowly upgrade to better hardware. It's not like there aren't options for the compute-constrained either. There are lots of Chinese models in the 3-32B range, and Gemma 3 is particularly good too. I will also point out that having three API-based providers deploying an impractically-large open-weights mod…
You said "Good." then wrote a nice stirring bit about how having a bad experience with a 1T model will force people to try 4B/32B models. That seems separate from the post it was replying to, about 1T param models. If it is intended to be a reply, it hand waves about how having a bad experience with it will teach them to buy more expensive hardware. Is that "Good."? The post points out that if people are taught they…
Had you posted this comment in the early 90s about linux instead of local models, it would have made about the same amount of sense but aged just as poorly as this comment will.
I'll remain here happily using 2.something tokens / second model.
Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model
#168Earlier quoted context omitted.
I'm of the personal opinion that it's quite reasonable for the creators to want attribution in case you manage to build a "successful product" off their work. The fact that it's a new or different license is a much smaller thing. A lot of open source, copyleft things already have attribution clauses. You're allowed commerical use of someone else's work already, regardless of scale. Attribution is a very benign ask.
I personally have no (or at least little) problem with attribution. As you say, quite a few licenses have some degree of attribution required. There's even a whole dedicated (and OSI approved) license who's raison d'être is about attribution: https://en.wikipedia.org/wiki/Common_Public_Attribution_Lice... What I'm saying, if I'm saying anything at all, is that it might have been better to pick one of these existing l…
You speak as if "license proliferation" is actually a problem.
But is it really?
Sure, it may make some licenses incompatible with each other, but that's basically equivalent to whining about somebody releasing their code in GPL and it can't be used in a project that uses MIT...
And your argument that the terms are "less understood" really doesn't matter. It's not like people know the Common Public Attribution License in and out either. (I'm going to argue that 99% devs don't even know the GPL well.) Poor drafting could be an issue, but I don't think this is the case here.
And on an ideological standpoint, I don't think people should be shamed into releasing their code under terms they aren't 100% comfortable with.
Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model
#169Re: Kimi K2 is a state-of-the-art mixture-of-experts (MoE) language model
#170Earlier quoted context omitted.
DDR3 workstation here - R1 generates at 1 token per second. In practice, this means that for complex queries, the speed of replying is closer to an email response than a chat message, but this is acceptable to me for confidential queries or queries where I need the model to be steerable. I can always hit the R1 API from a provider instead, if I want to. Given that R1 uses 37B active parameters (compared to 32B for K2…
That's pretty good. Are you running the real 600B+ parameter R1, or a distill, though?
The full thing, 671B. It loses some intelligence at 1.5 bit quantisation, but it's acceptable. I could actually go for around 3 bits if I max out my RAM, but I haven't done that yet.