Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

kimi.com

Post reply on HN
Print view View original

231–240 of 251 posts

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#231

Post by Ey7NFZ3P0nzAe »

codexon wrote: post #218
WhitneyLand wrote: post #129

Earlier quoted context omitted.

Its often pointed out in the first sentence of a comment how a model can be run at home, then (maybe) towards the end of the comment it’s mentioned how it’s quantized. Back when 4k movies needed expensive hardware, no one was saying they could play 4k on a home system, then later mentioning they actually scaled down the resolution to make it possible. The degree of quality loss is not often characterized. Which makes…

Didn't this paper demonstrate that you only need 1.58 bits to be equivalent to 16 bits in performance? https://arxiv.org/abs/2402.17764

This technique showed that there are ways during training to optimize weights to neatly quantize while remaining performant. This isn't a post training quantization like int4.

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#232

Post by segmondy »

mapkkk wrote: post #226
segmondy wrote: post #222

Earlier quoted context omitted.

I run KimiK2 at home, Most of it on system ram with a few layers offloaded to old 3090s. This is a cheap budget build. Kimi-K2-Thinking-UD-Q3_K_XL-00001-of-00010.gguf Generation - 5,231 tokens 604.63s 8.65 tokens/s

Could I trouble you for the specifics of your build? I'd love to see if it would be a viable upgrade for me. I currently have a 3970x with a bunch of 3090s.

4 3090s, epyc MB with 8 channel memory, 7352 cpu, slow 2400mhz ddr4 rams.

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#233

Post by greazy »

zozbot234 wrote: post #102
whizzter wrote: post #92

Earlier quoted context omitted.

Chinese state that maybe sees open collaboration as the way to nullify any US lead in the field, concurrently if the next "search-winner" is built upon their model the Chinese worldview that Taiwan belongs to China and Tiamen Square massacre never happened. Also their license says that if you have a big product you need to promote them, remember how Google "gave away" site searche widgets and that was perhaps one of…

> "Scaling to AGI/ASI" was always a fools errand Scaling depends on hardware, so cheaper hardware on a compute-per-watt basis only makes scaling easier. There is no clear definition of AGI/ASI but AI has already scaled to be quite useful.

> There is no clear definition of AGI

? Yes there's a clear defintion

https://en.wikipedia.org/wiki/Artificial_general_intelligenc...

I think you mean there is no agreed upon definition? Even that is dubious statement.

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#234

Post by petu »

WhitneyLand wrote: post #169
FuckButtons wrote: post #149

Earlier quoted context omitted.

From my own usage, the former is almost always better than the latter. Because it’s less like a lobotomy and more like a hangover, though I have run some quantized models that seem still drunk. Any model that I can run in 128 gb in full precision is far inferior to the models that I can just barely get to run after reap + quantization for actually useful work. I also read a paper a while back about improvements to mo…

Interesting. If this were the case however, why would labs go through the trouble of distilling their smaller models rather than releasing quantized versions of the flagships?

You can't quantize 1T model down to "flash" model speed/token price. 4bpw is about the limit of reasonable quantization, so 2-4x (fp8/16 -> 4bpw) weight size reduction. Easier to serve, sure, but maybe not offer as free tier cheap.

With distillation you're training new model, so size of it is arbitrary, say 1T -> 20B (50x) reduction which also can be quantized. AFAIK distillation is also simply faster/cheaper than training from scratch.

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#235

Post by WhitneyLand »

Ey7NFZ3P0nzAe wrote: post #231
codexon wrote: post #218

Earlier quoted context omitted.

Didn't this paper demonstrate that you only need 1.58 bits to be equivalent to 16 bits in performance? https://arxiv.org/abs/2402.17764

This technique showed that there are ways during training to optimize weights to neatly quantize while remaining performant. This isn't a post training quantization like int4.

For Kimi quantization is part of the training also. Specifically they say they use QAT, quantization aware training.

That doesn't mean training with all integer math, but certain tricks are used to specifically plan for the end weight size. I.e. fake quantization nodes are inserted to simulate int4.

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#236

Post by flexagoon »

Reubend wrote: post #13

I've read several people say that Kimi K2 has a better "emotional intelligence" than other models. I'll be interested to see whether K2.5 continues or even improves on that.

I love the Kimi response style. It's much more concise, without all the unnecessary "great question!"s and other annoying AI stuff

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#237

Post by vuldin »

Imanari wrote: post #108
vinhnx wrote: post #42

One thing caught my eyes is that besides K2.5 model, Moonshot AI also launched Kimi Code ( https://www.kimi.com/code ), evolved from Kimi CLI. It is a terminal coding agent, I've been used it last month with Kimi subscription, it is capable agent with stable harness. GitHub: https://github.com/MoonshotAI/kimi-cli

How does it fare against CC?

Anecdotally, I've cancelled my Claude Code subscription after using Kimi K2.5 and Kimi CLI for the last few days. It's handled everything I've thrown at it. It is slower at the moment, but I expect that will improve.

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#238

Post by alimbada »

dabockster wrote: post #195
johndough wrote: post #81

Earlier quoted context omitted.

The model absolutely can be run at home. There even is a big community around running large models locally: https://www.reddit.com/r/LocalLLaMA/ The cheapest way is to stream it from a fast SSD, but it will be quite slow (one token every few seconds). The next step up is an old server with lots of RAM and many memory channels with maybe a GPU thrown in for faster prompt processing (low two digits tokens/second). At t…

You can run AI models on unified/shared memory specifically on Windows, not Linux (unfortunately). It uses the same memory sharing system that Microsoft originally had built for gaming when a game would run out of vram. If you: - have an i5 or better or equivalent manufactured within the last 5-7 years - have an nvidia consumer gaming GPU (RTX 3000 series or better) with at least 8 GB vram - have at least 32 GB syste…

> Now, the reason why it won't work on Linux is that the Linux kernel and Linux distros both leave that unified memory capability up to the GPU driver to implement. Which Nvidia hasn't done yet. You can code it somewhat into source code, but it's still super unstable and flaky from what I've read.

So it should work with an AMD GPU?

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#239

Post by michaelcampbell »

conception wrote: post #123
Alifatisk wrote: post #53

Have you all noted that the latest releases (Qwen3 max thinking, now Kimi k2.5) from Chinese companies are benching against Claude opus now and not Sonnet? They are truly catching up, almost at the same pace?

https://clocks.brianmoore.com K2 is one of the only models to nail the clock face test as well. It’s a great model.

It's better than most, but not 100%. As I see this the clock hands are all correct, but the numbers only go 1-8.

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#240

Post by kristianp »

storystarling wrote: post #202
wongarsu wrote: post #78

Earlier quoted context omitted.

Which conveniently fits on one 8xH100 machine. With 100-200 GB left over for overhead, kv-cache, etc.

The unit economics seem pretty rough though. You're locking up 8xH100s for the compute of ~32B active parameters. I guess memory is the bottleneck but hard to see how the margins work on that.

Yes, it only makes sense economically if you have batching over many users.
Post reply on HN