Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K2 1T model runs on 2 512GB M3 Ultras

twitter.com

Post reply on HN
Print view View original

51–60 of 125 posts

Re: Kimi K2 1T model runs on 2 512GB M3 Ultras

#51

Post by cubefox »

NitpickLawyer wrote: post #45
websiteapi wrote: post #7

I get tempted to buy a couple of these, but I just feel like the amortization doesn’t make sense yet. Surely in the next few years this will be orders of magnitude cheaper.

Before committing to purchasing two of these, you should look at the true speeds that few people post. Not just the "it works". We're at a point where we can run these very large models "at home", and it is great! But true usage is now with very large contexts, both in prompt processing, and token generations. Whatever speeds these models get at "0" context is very different than what they get at "useful" context, es…

DeepSeek-v3.2 should be be better for long context because it is using (near linear) sparse attention.

Re: Kimi K2 1T model runs on 2 512GB M3 Ultras

#52

Post by Bolwin »

A_D_E_P_T wrote: post #3

Kimi K2 is a really weird model, just in general. It's not nearly as smart as Opus 4.5 or 5.2-Pro or whatever, but it has a very distinct writing style and also a much more direct "interpersonal" style. As a writer of very-short-form stuff like emails, it's probably the best model available right now. As a chatbot, it's the only one that seems to really relish calling you out on mistakes or nonsense, and it doesn't h…

In their AMA moonshot said it was mainly finetuning

Re: Kimi K2 1T model runs on 2 512GB M3 Ultras

#53

Post by rubymamis »

What benchmarks are good these days? I generally just try different models on Cursor, but most of the open weight models aren't available there (Deepseak v3.2, Kimi K2 has some problems with formatting, and many others are missing) so I'd be curious to see some benchmarks - especially for non-web stuff (C++, Rust, etc).

Re: Kimi K2 1T model runs on 2 512GB M3 Ultras

#54

Post by IgorPartola »

stingraycharles wrote: post #36
IgorPartola wrote: post #21

Earlier quoted context omitted.

Full instruction following looks like monkey’s paw/malicious compliance. A good way to eliminate a bug from a codebase is to delete the codebase, that type of thing. You want the model to have enough creative freedom to solve the problem otherwise you are just coding using an imprecise language spec. I know what you mean: a lot of my prompts include “never use em-dashes” but all models forget this sooner or later. Bu…

My point is that it’s better that the model asks questions to better understand what’s going on before pushing back.

Agreed. With Claude Code I will often specify the feature I want to develop, then tell it to summarize the plan for me, give me its opinion on the plan, and ask questions before it does anything. This works very well. Often times it actually catches some piece I didn’t consider and this almost always results in usable code or code that is close enough that Claude can fix after I review what it did and point out problems.

Re: Kimi K2 1T model runs on 2 512GB M3 Ultras

#56

Post by CamperBob2 »

3abiton wrote: post #50
A_D_E_P_T wrote: post #3

Kimi K2 is a really weird model, just in general. It's not nearly as smart as Opus 4.5 or 5.2-Pro or whatever, but it has a very distinct writing style and also a much more direct "interpersonal" style. As a writer of very-short-form stuff like emails, it's probably the best model available right now. As a chatbot, it's the only one that seems to really relish calling you out on mistakes or nonsense, and it doesn't h…

> I get the feeling that it was trained very differently from the other models It's actually based on a deepseek architecture just bigger size experts if I recall correctly.

As far as I'm aware, they all are. There are only five important foundation models in play -- Gemini, GPT, X.ai, Claude, and Deepseek. (edit: forgot Claude)

Everything from China is downstream of Deepseek, which some have argued is basically a protege of ChatGPT.

Re: Kimi K2 1T model runs on 2 512GB M3 Ultras

#57

Post by Alifatisk »

A_D_E_P_T wrote: post #3

Kimi K2 is a really weird model, just in general. It's not nearly as smart as Opus 4.5 or 5.2-Pro or whatever, but it has a very distinct writing style and also a much more direct "interpersonal" style. As a writer of very-short-form stuff like emails, it's probably the best model available right now. As a chatbot, it's the only one that seems to really relish calling you out on mistakes or nonsense, and it doesn't h…

> As a writer of very-short-form stuff like emails, it's probably the best model available right now.

This is exactly my feeling with Kimi K2, it's unique in this regard, the only one that comes close is Gemini 3 pro, otherwise, no other model has been this good at helping out with communication.

It has such a good understanding with "emotional intelligence" (?), reading signals in messages, understanding intentions, taking human factors into consideration and social norms and trends when helping out with formulating a message.

I don't exactly know what Moonshot did during training but they succeeded with a unique trait on this model. This area deserves more highlight in my opinion.

I saw someone linking to EQ-bench which is about emotional intelligence in LLMs, looking at it, Kimi is #1. So this kind of confirms my feeling.

Link: https://eqbench.com

Re: Kimi K2 1T model runs on 2 512GB M3 Ultras

#59

Post by kingstnap »

CamperBob2 wrote: post #56
3abiton wrote: post #50

Earlier quoted context omitted.

> I get the feeling that it was trained very differently from the other models It's actually based on a deepseek architecture just bigger size experts if I recall correctly.

As far as I'm aware, they all are. There are only five important foundation models in play -- Gemini, GPT, X.ai, Claude, and Deepseek. (edit: forgot Claude) Everything from China is downstream of Deepseek, which some have argued is basically a protege of ChatGPT.

Not true, Qwen from Alibaba does lots of random architectures.

Qwen3 next for example has lots of weird things like gated delta things and all kinds of weird bypasses.

https://qwen.ai/blog?id=4074cca80393150c248e508aa62983f9cb7d...

Re: Kimi K2 1T model runs on 2 512GB M3 Ultras

#60

Post by swores »

kingstnap wrote: post #59
CamperBob2 wrote: post #56

Earlier quoted context omitted.

As far as I'm aware, they all are. There are only five important foundation models in play -- Gemini, GPT, X.ai, Claude, and Deepseek. (edit: forgot Claude) Everything from China is downstream of Deepseek, which some have argued is basically a protege of ChatGPT.

Not true, Qwen from Alibaba does lots of random architectures. Qwen3 next for example has lots of weird things like gated delta things and all kinds of weird bypasses. https://qwen.ai/blog?id=4074cca80393150c248e508aa62983f9cb7d...

Agree with you over OP - as well as Qwen there's others like Mistral, Meta's Llama, and from China there's the likes of Baidu ERNIE, ByteDance Doubao, and Zhipu GLM. Probably others too.

Even if all of these were considered worse than the "only 5" on OP's list (which I don't believe to be the case), the scene is still far too young and volatile to look at a ranking at any one point in time and say that if X is better than Y today then it definitely will be in 3 months time, yet alone in a year or two.

Post reply on HN