101–110 of 146 posts
Earlier quoted context omitted.
Can you share how you're running it?
> Can you share how you're running it? Not OP, but I've been running it through Kagi [1]. Their AI offering is probably the best-kept secret in the market. [1] https://help.kagi.com/kagi/ai/assistant.html
Seems that K2.5 has lost a lot of the personality from K2 unfortunately, talks in more ChatGPT/Gemini/C-3PO style now. It's not explictly bad, I'm sure most people won't care but it was something that made it unique so it's a shame to see it go. examples to illustrate https://www.kimi.com/share/19c115d6-6402-87d5-8000-000062fec... (K2.5) https://www.kimi.com/share/19c11615-8a92-89cb-8000-000063ee6... (K2)
Earlier quoted context omitted.
Just pick up any >240GB VRAM GPU off your local BestBuy to run a quantized version. > The full Kimi K2.5 model is 630GB and typically requires at least 4× H200 GPUs.
You could run the full, unquantized model at high speed with 8 RTX 6000 Blackwell boards. I don't see a way to put together a decent system of that scale for less than $100K, given RAM and SSD prices. A system with 4x H200s would cost more like $200K.
I've been using this model (as a coding agent) for the past few days, and it's the first time I've felt that an open source model really competes with the big labs. So far it's been able to handle most things I've thrown at it. I'm almost hesitant to say that this is as good as Opus.
Earlier quoted context omitted.
You could buy five Strix Halo systems at $2000 each, network them and run it. Rough estimage: 12.5:2.2 so you should get around 5.5 tokens/s.
Is the software/drivers for networking LLMs on Strix Halo there yet? I was under the impression a few weeks ago that it's veeeery early stages and terribly slow.
Sorry if this is an easy-answerable question - but by open we can download this and use totally offline if now or in the future if we have hardware capable? Seems like a great thing to archive if the world falls apart (said half-jokingly)
Also, that's with no context, so it would be slower as it filled (I don't think K2.5 uses the Kimi-Linear KDA attention mechanism, so it's sub-quadratic but not their lowest).
Seems that K2.5 has lost a lot of the personality from K2 unfortunately, talks in more ChatGPT/Gemini/C-3PO style now. It's not explictly bad, I'm sure most people won't care but it was something that made it unique so it's a shame to see it go. examples to illustrate https://www.kimi.com/share/19c115d6-6402-87d5-8000-000062fec... (K2.5) https://www.kimi.com/share/19c11615-8a92-89cb-8000-000063ee6... (K2)
I've been using this model (as a coding agent) for the past few days, and it's the first time I've felt that an open source model really competes with the big labs. So far it's been able to handle most things I've thrown at it. I'm almost hesitant to say that this is as good as Opus.
Did you use Kimi Code or some other harness? I used it with OpenCode and it was bumbling around through some tasks that Claude handles with ease.