Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

kimi.com

Post reply on HN
Print view View original

221–230 of 251 posts

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#221

Post by WhitneyLand »

codexon wrote: post #218
WhitneyLand wrote: post #129

Earlier quoted context omitted.

Its often pointed out in the first sentence of a comment how a model can be run at home, then (maybe) towards the end of the comment it’s mentioned how it’s quantized. Back when 4k movies needed expensive hardware, no one was saying they could play 4k on a home system, then later mentioning they actually scaled down the resolution to make it possible. The degree of quality loss is not often characterized. Which makes…

Didn't this paper demonstrate that you only need 1.58 bits to be equivalent to 16 bits in performance? https://arxiv.org/abs/2402.17764

Iirc the paper was solid, but it still hasn’t been adopted/proven out at large scale. Harder to adapt hardware and code kernels to something like this compared to int4.

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#222

Post by segmondy »

endymi0n wrote: post #45
Tepix wrote: post #8

Huggingface Link: https://huggingface.co/moonshotai/Kimi-K2.5 1T parameters, 32b active parameters. License: MIT with the following modification: Our only modification part is that, if the Software (or any derivative works thereof) is used for any of your commercial products or services that have more than 100 million monthly active users, or more than 20 million US dollars (or equivalent in other currencies) in mont…

One. Trillion. Even on native int4 that’s… half a terabyte of vram?! Technical awe at this marvel aside that cracks the 50th percentile of HLE, the snarky part of me says there’s only half the danger in giving something away nobody can run at home anyway…

I run KimiK2 at home, Most of it on system ram with a few layers offloaded to old 3090s. This is a cheap budget build.

Kimi-K2-Thinking-UD-Q3_K_XL-00001-of-00010.gguf Generation - 5,231 tokens 604.63s 8.65 tokens/s

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#223

Post by segmondy »

bertili wrote: post #34

The "Deepseek moment" is just one year ago today! Coincidence or not, let's just marvel for a second over this amount of magic/technology that's being given away for free... and how liberating and different this is than OpenAI and others that were closed to "protect us all".

There's been so many moments that folks not really heavy into LLM have missed, DeepSeekR1 was great, but so was all the "incremental" improvements, v3-0324, v3.1, v3.1-terminus, and now v3.2-speciale. With that this is the 3rd great Kimi model, then GLM has been awesome, since 4.5, with 4.5, 4.5-air, 4.6, 4.7 and now 4.7 flash. Minimax-M2 has also been making waves lately. ... and i'm just talking about the Chinese model without adding the 10+ Qwen models. Outside of Chinese models, mistral-small/devstral, gemma-27b-it, gpt-oss-120b, seed-os have been great, and I'm still talking about just LLM, not image, audio or special domain models like deepseek-prover and deepseek-math. It's really a marvel what we have at home. I cancelled OpenAI and Anthropic subscription 2 years ago once they started calling for regulation of open models and I haven't missed them one bit.

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#225

Post by esafak »

hex4def6 wrote: post #203
esafak wrote: post #143

Earlier quoted context omitted.

Hear, hear. Even if the model fits, a few tokens per second make no sense. Time is money too.

If I can start an agent and be able to walk away for 8 hours, and be confident it's 'smart' enough to complete a task unattended, that's still useful. At 3 tk/s, that's still 100-150 pages of a book, give or take.

True, that's still faster than a human, but they're not nearly that reliable yet.

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#226

Post by mapkkk »

segmondy wrote: post #222
endymi0n wrote: post #45

Earlier quoted context omitted.

One. Trillion. Even on native int4 that’s… half a terabyte of vram?! Technical awe at this marvel aside that cracks the 50th percentile of HLE, the snarky part of me says there’s only half the danger in giving something away nobody can run at home anyway…

I run KimiK2 at home, Most of it on system ram with a few layers offloaded to old 3090s. This is a cheap budget build. Kimi-K2-Thinking-UD-Q3_K_XL-00001-of-00010.gguf Generation - 5,231 tokens 604.63s 8.65 tokens/s

Could I trouble you for the specifics of your build? I'd love to see if it would be a viable upgrade for me.

I currently have a 3970x with a bunch of 3090s.

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#227

Post by mholm »

PlatoIsADisease wrote: post #213
simonw wrote: post #194

Earlier quoted context omitted.

I don't think Awni should be dismissed as a "marketing account" - they're an engineer at Apple who's been driving the MLX project for a couple of years now, they've earned a lot of respect from me.

Given how secretive Apple is, oh my, its super duper marketing account.

Jeff Geerling and a few others also got access to similarly specced mac clusters. They replicated this performance.

The tooling involved has improved significantly over the past year.

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#228

Post by [deleted] »

1dom wrote: post #148
zamadatix wrote: post #137

Earlier quoted context omitted.

Local LLMs are just LLMs people run locally. It's not a definition of size, feature set, or what's most popular. What the "real" value is for local LLMs will depend on each person you ask. The person who runs small local LLMs will tell you the real value is in small models, the person who runs large local LLMs will tell you it's large ones, those who use cloud will say the value is in shared compute, and those who do…

> LLMs which the weights aren't available are an example of when it's not local LLMs, not when the model happens to be large. I agree. My point was that most aren't thinking of models this large when they're talking about local LLMs. That's what I said, right? This is supported by the download counts on hf: the most downloaded local models are significantly smaller than 1tln, normally 1 - 12bln. I'm not sure I unders…

[deleted]

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#229

Post by [deleted] »

1dom wrote: post #148
zamadatix wrote: post #137

Earlier quoted context omitted.

Local LLMs are just LLMs people run locally. It's not a definition of size, feature set, or what's most popular. What the "real" value is for local LLMs will depend on each person you ask. The person who runs small local LLMs will tell you the real value is in small models, the person who runs large local LLMs will tell you it's large ones, those who use cloud will say the value is in shared compute, and those who do…

> LLMs which the weights aren't available are an example of when it's not local LLMs, not when the model happens to be large. I agree. My point was that most aren't thinking of models this large when they're talking about local LLMs. That's what I said, right? This is supported by the download counts on hf: the most downloaded local models are significantly smaller than 1tln, normally 1 - 12bln. I'm not sure I unders…

[deleted]

Re: Kimi Released Kimi K2.5, Open-Source Visual SOTA-Agentic Model

#230

Post by zamadatix »

1dom wrote: post #148
zamadatix wrote: post #137

Earlier quoted context omitted.

Local LLMs are just LLMs people run locally. It's not a definition of size, feature set, or what's most popular. What the "real" value is for local LLMs will depend on each person you ask. The person who runs small local LLMs will tell you the real value is in small models, the person who runs large local LLMs will tell you it's large ones, those who use cloud will say the value is in shared compute, and those who do…

> LLMs which the weights aren't available are an example of when it's not local LLMs, not when the model happens to be large. I agree. My point was that most aren't thinking of models this large when they're talking about local LLMs. That's what I said, right? This is supported by the download counts on hf: the most downloaded local models are significantly smaller than 1tln, normally 1 - 12bln. I'm not sure I unders…

Mostly a "We know local LLMs as being this, and all of the mentioned variants of this can provide real value regardless of which is most commonly referenced" point. I.e. large local LLMs aren't only something people mess with, they often provide a lot of value for a relative few people rather than a little value for a relative lot of people as small local LLMs do. Who thinks which modality and type brings the most value is largely a matter of opinion of the user getting the value, not just the option which runs on consumer hardware or etc alone.

You're of course accurate that smaller LLMs are more commonly deployed, it's just not the part I was really responding to.

Post reply on HN