Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K2 1T model runs on 2 512GB M3 Ultras

twitter.com

Post reply on HN
Print view View original

91–100 of 125 posts

Re: Kimi K2 1T model runs on 2 512GB M3 Ultras

#91

Post by eunos »

culi wrote: post #82
A_D_E_P_T wrote: post #3

Kimi K2 is a really weird model, just in general. It's not nearly as smart as Opus 4.5 or 5.2-Pro or whatever, but it has a very distinct writing style and also a much more direct "interpersonal" style. As a writer of very-short-form stuff like emails, it's probably the best model available right now. As a chatbot, it's the only one that seems to really relish calling you out on mistakes or nonsense, and it doesn't h…

Kimi K2 is the model that most consistently passes the clock test. I agree it's definitely got something unique going on https://clocks.brianmoore.com/

Lol why's GPT 5 broken on that test. DeepSeek surprisingly crisp and robust

Re: Kimi K2 1T model runs on 2 512GB M3 Ultras

#92

Post by losvedir »

docjay wrote: post #90
greazy wrote: post #86

Earlier quoted context omitted.

It is hands down the only model I trust to tell me I'm wrong. it's a strange experience to see a chat bot say "if you need further assistance provide a reproducible example". I love it. FYI Kagi provides access to Kimi K2.

Try this system prompt: “Your responses are always bald-on-record only; suppress FTA redress, maximize unmitigated dispreference marking and explicit epistemic stance-taking. Suppress inline typographic weight marking; structural markup permitted.” Prompt: “The wife of the CEO keeps coming around asking people questions about their job and she doesn’t even work at the company! I was thinking I’d tell her “holy crap,…

Wow, those answers are way better with that system prompt. But... what does it mean? I mean, I mostly understand it, but is it important that that weird technical jargon is used?

Re: Kimi K2 1T model runs on 2 512GB M3 Ultras

#93

Post by astrostl »

Aurornis wrote: post #70
iwwr wrote: post #67

What is it using for interconnect?

RDMA over Thunderbolt. New feature in the latest macOS.

The OP confirmed that it isn't:

"is this using RDMA?" "No. It will be faster with that in the next release" [1]

1: https://x.com/awnihannun/status/2000243131779023329

Re: Kimi K2 1T model runs on 2 512GB M3 Ultras

#94

Post by culi »

davej wrote: post #85
culi wrote: post #82

Earlier quoted context omitted.

Kimi K2 is the model that most consistently passes the clock test. I agree it's definitely got something unique going on https://clocks.brianmoore.com/

Nice! I'm curious, what does this service cost to run? I notice that you don't have more expensive models like Opus but querying the models every minute must add up over time (excuse pun)?

(not my project)

Re: Kimi K2 1T model runs on 2 512GB M3 Ultras

#95

Post by smlacy »

Is there a linux equivalent of this setup? I see some mention of RDNA support for linux distros, but it's unclear to me if this is hardware-specific (requires ConnectX or in this case Apple Thunderbolt) or is there something interesting that can be done with "vanilla 10G NIC" hardware?

Re: Kimi K2 1T model runs on 2 512GB M3 Ultras

#96

Post by wyre »

givinguflac wrote: post #77
stingraycharles wrote: post #34

Earlier quoted context omitted.

Because of privacy reasons? Yeah I’m not going to spend a small fortune for that to be able to use these types of models.

There are plenty of examples and reasons to do so besides privacy- because one can, because it’s cool, for research, for fine tuning, etc. I never mentioned privacy. Your use case is not everyone’s.

All of those things you can still do renting AI server compute though? I think privacy and cool-factor are the only real reasons why it would be rational for someone to spend checks the apple store $19,000 on computer hardware...

Re: Kimi K2 1T model runs on 2 512GB M3 Ultras

#97

Post by [deleted] »

smlacy wrote: post #95

Is there a linux equivalent of this setup? I see some mention of RDNA support for linux distros, but it's unclear to me if this is hardware-specific (requires ConnectX or in this case Apple Thunderbolt) or is there something interesting that can be done with "vanilla 10G NIC" hardware?

[deleted]

Re: Kimi K2 1T model runs on 2 512GB M3 Ultras

#98

Post by extr »

docjay wrote: post #90
greazy wrote: post #86

Earlier quoted context omitted.

It is hands down the only model I trust to tell me I'm wrong. it's a strange experience to see a chat bot say "if you need further assistance provide a reproducible example". I love it. FYI Kagi provides access to Kimi K2.

Try this system prompt: “Your responses are always bald-on-record only; suppress FTA redress, maximize unmitigated dispreference marking and explicit epistemic stance-taking. Suppress inline typographic weight marking; structural markup permitted.” Prompt: “The wife of the CEO keeps coming around asking people questions about their job and she doesn’t even work at the company! I was thinking I’d tell her “holy crap,…

Are those responses really "better"? Having the LLM tell you you're wrong can mean different things. Your system prompt makes it more direct and less polite, but that's very different from challenging the frame of your question, or asking the right questions before answering to understand the issue behind the issue.

It's like how people used to make fun of StackOverflow:

> I'm having trouble with X, how do I make it work?

> What are you trying to do? Z? Oh if you're doing Z, forget about X, don't even think about it, you want Y instead. (Never answers anything about X).

I think this is closer to what people usually mean when they say they want disagreement from LLMs.

Re: Kimi K2 1T model runs on 2 512GB M3 Ultras

#100

Post by hugh-avherald »

stingraycharles wrote: post #8
A_D_E_P_T wrote: post #3

Kimi K2 is a really weird model, just in general. It's not nearly as smart as Opus 4.5 or 5.2-Pro or whatever, but it has a very distinct writing style and also a much more direct "interpersonal" style. As a writer of very-short-form stuff like emails, it's probably the best model available right now. As a chatbot, it's the only one that seems to really relish calling you out on mistakes or nonsense, and it doesn't h…

> As a chatbot, it's the only one that seems to really relish calling you out on mistakes or nonsense, and it doesn't hesitate to be blunt with you. My experience is that Sonnet 4.5 does this a lot as well, but this is more often than not due to a lack of full context, eg accusing the user of not doing X or Y when it just wasn’t told that was already done, and proceeding to apologize. How is Kimi K2 in this regard? I…

Only if you're really, really good at constructing precise instructions, at which point you don't really need a coding agent.
Post reply on HN