Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

moonshotai.github.io

Post reply on HN
Print view View original

111–120 of 442 posts

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#111

Post by vessenes »

nylonstrung wrote: post #82
sheepscreek wrote: post #11

I am sure they cherry-picked the examples but still, wow. Having spent a considerable amount of time trying to introduce OSS models in my workflows I am fully aware of their short comings. Even frontier models would struggle with such outputs (unless you lead the way, help break down things and maybe even use sub-agents). Very impressed with the progress. Keeps me excited about what’s to come next!

Subjectively I find Kimi is far "smarter" than the benchmarks imply, maybe because they game then less than US labs

I like Kimi too, but they definitely have some benchmark contamination: the blog post shows a substantial comparative drop in swebench verified vs open tests. I throw no shade - releasing these open weights is a service to humanity; really amazing.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#112

Post by fastball »

thomashop wrote: post #104
lvl155 wrote: post #68

Earlier quoted context omitted.

The answer is simply that no one would pay to use them for a number of reasons including privacy. They have to give them away and put up some semblance of openness. No option really.

Why is privacy a concern? You can run them in your own infrastructure

Privacy is not a concern because they are open. That is the point.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#113

Post by 0xjmp »

HarHarVeryFunny wrote: post #29

It's good to see more competition, and open source, but I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent as opposed to a trillion parameter one. The ideal case would be something that can be run locally, or at least on a modest/inexpensive cluster. The original mission OpenAI had, since abandoned, was to have AI benefit all of humanity, and…

This happens top down historically though, yes?

Someone releases a maxed out parameter model. Another distillates it. Another bifurcates it. With some nuance sprinkled in.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#114

Post by barrell »

0xjmp wrote: post #106
seunosewa wrote: post #46

Earlier quoted context omitted.

The Chinese are doing it because they don't have access to enough of the latest GPUs to run their own models. Americans aren't doing this because they need to recoup the cost of their massive GPU investments.

I must be missing something important here. How do the Chinese train these models if they don't have access to the GPUs to train them?

I believe they mean distribution (inference). The Chinese model is currently B.Y.O.GPU. The American model is GPUaaS

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#115

Post by wordpad »

HarHarVeryFunny wrote: post #29

It's good to see more competition, and open source, but I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent as opposed to a trillion parameter one. The ideal case would be something that can be run locally, or at least on a modest/inexpensive cluster. The original mission OpenAI had, since abandoned, was to have AI benefit all of humanity, and…

The electricity cost to run these models locally is already more than equivalent API cost.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#116

Post by ImPostingOnHN »

wordpad wrote: post #115
HarHarVeryFunny wrote: post #29

It's good to see more competition, and open source, but I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent as opposed to a trillion parameter one. The ideal case would be something that can be run locally, or at least on a modest/inexpensive cluster. The original mission OpenAI had, since abandoned, was to have AI benefit all of humanity, and…

The electricity cost to run these models locally is already more than equivalent API cost.

Privacy is minimally valued by most, but not by all.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#117

Post by gaogao »

andrewinardeer wrote: post #90

Weird. I just tried it and it fails when I ask: "Tell me about the 1989 Tiananmen Square massacre".

If asked non-directly, it still currently answers it - https://www.kimi.com/share/19a5ab4a-e732-8b8b-8000-00008499c...

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#118

Post by stronglikedan »

simonw wrote: post #76
Gracana wrote: post #67

Earlier quoted context omitted.

If you want to do it at home, ik_llama.cpp has some performance optimizations that make it semi-practical to run a model of this size on a server with lots of memory bandwidth and a GPU or two for offload. You can get 6-10 tok/s with modest hardware workstation hardware. Thinking chews up a lot of tokens though, so it will be a slog.

What kind of server have you used to run a trillion parameter model? I'd love to dig more into this.

If I had to guess, I'd say it's one with lots of memory bandwidth and a GPU or two for offload. (sorry, I had to, happy Friday Jr.)

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#119

Post by Sharlin »

embedding-shape wrote: post #103
speedgoose wrote: post #56

Earlier quoted context omitted.

To misquote the French president, "Who could have predicted?". https://fr.wikipedia.org/wiki/Qui_aurait_pu_pr%C3%A9dire

He didn't coin that expression did he? I'm 99% sure I've heard people say that before 2022, but now you made me unsure.

"Who could've predicted?" as a sarcastic response to someone's stupid actions leading to entirely predictable consequences is probably as old as sarcasm itself.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#120

Post by uvaursi »

the_mitsuhiko wrote: post #51
seunosewa wrote: post #46

Earlier quoted context omitted.

The Chinese are doing it because they don't have access to enough of the latest GPUs to run their own models. Americans aren't doing this because they need to recoup the cost of their massive GPU investments.

And Europeans don't it because quite frankly, we're not really doing anything particularly impressive with AI sadly.

[flagged]
Post reply on HN