Kimi K2 1T model runs on 2 512GB M3 Ultras
twitter.com
Kimi K2 1T model runs on 2 512GB M3 Ultras
1–10 of 125 posts
Re: Kimi K2 1T model runs on 2 512GB M3 Ultras
#2You should mention that it is 4bit quant. Still very impressive!
Re: Kimi K2 1T model runs on 2 512GB M3 Ultras
#3Kimi K2 is a really weird model, just in general.
It's not nearly as smart as Opus 4.5 or 5.2-Pro or whatever, but it has a very distinct writing style and also a much more direct "interpersonal" style. As a writer of very-short-form stuff like emails, it's probably the best model available right now. As a chatbot, it's the only one that seems to really relish calling you out on mistakes or nonsense, and it doesn't hesitate to be blunt with you.
I get the feeling that it was trained very differently from the other models, which makes it situationally useful even if it's not very good for data analysis or working through complex questions. For instance, as it's both a good prose stylist and very direct/blunt, it's an extremely good editor.
I like it enough that I actually pay for a Kimi subscription.
Re: Kimi K2 1T model runs on 2 512GB M3 Ultras
#4Kimi K2 is a very impressive model! It's particularly un-obsequious, which makes it useful for actually checking your reasoning on things.
Some especially older ChatGPT models will tell you that everything you say is fantastic and great. Kimi -on the other hand- doesn't mind taking a detour to question your intelligence and likely your entire ancestry if you ask it to be brutal.
Re: Kimi K2 1T model runs on 2 512GB M3 Ultras
#5Kimi K2 is a really weird model, just in general. It's not nearly as smart as Opus 4.5 or 5.2-Pro or whatever, but it has a very distinct writing style and also a much more direct "interpersonal" style. As a writer of very-short-form stuff like emails, it's probably the best model available right now. As a chatbot, it's the only one that seems to really relish calling you out on mistakes or nonsense, and it doesn't h…
It's also the only model that consistently nails my favorite AI benchmark: https://clocks.brianmoore.com/
Re: Kimi K2 1T model runs on 2 512GB M3 Ultras
#6Kimi K2 is a really weird model, just in general. It's not nearly as smart as Opus 4.5 or 5.2-Pro or whatever, but it has a very distinct writing style and also a much more direct "interpersonal" style. As a writer of very-short-form stuff like emails, it's probably the best model available right now. As a chatbot, it's the only one that seems to really relish calling you out on mistakes or nonsense, and it doesn't h…
Speaking of weird. I feel like Kimi is a shoggoth with its tentacles in a man-bun. If that makes any sense.
Re: Kimi K2 1T model runs on 2 512GB M3 Ultras
#7I get tempted to buy a couple of these, but I just feel like the amortization doesn’t make sense yet. Surely in the next few years this will be orders of magnitude cheaper.
Re: Kimi K2 1T model runs on 2 512GB M3 Ultras
#8Kimi K2 is a really weird model, just in general. It's not nearly as smart as Opus 4.5 or 5.2-Pro or whatever, but it has a very distinct writing style and also a much more direct "interpersonal" style. As a writer of very-short-form stuff like emails, it's probably the best model available right now. As a chatbot, it's the only one that seems to really relish calling you out on mistakes or nonsense, and it doesn't h…
> As a chatbot, it's the only one that seems to really relish calling you out on mistakes or nonsense, and it doesn't hesitate to be blunt with you.
My experience is that Sonnet 4.5 does this a lot as well, but this is more often than not due to a lack of full context, eg accusing the user of not doing X or Y when it just wasn’t told that was already done, and proceeding to apologize.
How is Kimi K2 in this regard?
Isn’t “instruction following” the most important thing you’d want out of a model in general, and a model pushing back more likely than not being wrong?
Re: Kimi K2 1T model runs on 2 512GB M3 Ultras
#9Kimi K2 is a really weird model, just in general. It's not nearly as smart as Opus 4.5 or 5.2-Pro or whatever, but it has a very distinct writing style and also a much more direct "interpersonal" style. As a writer of very-short-form stuff like emails, it's probably the best model available right now. As a chatbot, it's the only one that seems to really relish calling you out on mistakes or nonsense, and it doesn't h…
> As a chatbot, it's the only one that seems to really relish calling you out on mistakes or nonsense, and it doesn't hesitate to be blunt with you. My experience is that Sonnet 4.5 does this a lot as well, but this is more often than not due to a lack of full context, eg accusing the user of not doing X or Y when it just wasn’t told that was already done, and proceeding to apologize. How is Kimi K2 in this regard? I…
> Isn’t “instruction following” the most important thing you’d want out of a model in general,
No. And for the same reason that pure "instruction following" in humans is considered a form of protest/sabotage.
Re: Kimi K2 1T model runs on 2 512GB M3 Ultras
#10I get tempted to buy a couple of these, but I just feel like the amortization doesn’t make sense yet. Surely in the next few years this will be orders of magnitude cheaper.
I don’t think it will ever make sense; you can buy so much cloud based usage for this type of price.
From my perspective, the biggest problem is that I am just not going to be using it 24/7. Which means I’m not getting nearly as much value out of it as the cloud based vendors do from their hardware.
Last but not least, if I want to run queries against open source models, I prefer to use a provider like Groq or Cerebras as it’s extremely convenient to have the query results nearly instantly.