Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

moonshotai.github.io

Post reply on HN
Print view View original

291–300 of 442 posts

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#291

Post by Der_Einzige »

simonw wrote: post #288
electroglyph wrote: post #281

Earlier quoted context omitted.

i assume that $4.6 mil is just the cost of the electricity?

Hard to be sure because the source of that information isn't known, but generally when people talk about training costs like this they include more than just the electricity but exclude staffing costs. Other reported training costs tend to include rental of the cloud hardware (or equivalent if the hardware is owned by the company), e.g. NVIDIA H100s are sometimes priced out in cost-per-hour.

Citation needed on "generally when people talk about training costs like this they include more than just the electricity but exclude staffing costs".

It would be simply wrong to exclude the staffing costs. When each engineer costs well over 1 million USD in total costs year over year, you sure as hell account for them.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#292

Post by Der_Einzige »

barrell wrote: post #114
0xjmp wrote: post #106

Earlier quoted context omitted.

I must be missing something important here. How do the Chinese train these models if they don't have access to the GPUs to train them?

I believe they mean distribution (inference). The Chinese model is currently B.Y.O.GPU. The American model is GPUaaS

That's super wrong. A lot of why people flipped out about Deepseek V3 is because of how cheap and how fast their GPUaaS model is.

There is so much misinformation both on HN, and in this very thread about LLMs and GPUs and cloud and it's exhausting trying to call it out all the time - especially when it's happening from folks who are considered "respected" in the field.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#293

Post by Der_Einzige »

brandall10 wrote: post #121
gordonhart wrote: post #62

Earlier quoted context omitted.

It’s more fair to say they gave up after the Llama 4 disaster.

Also, the Meta AI 'team' is currently retooling so they can put something together with a handful of Zuck-picked experts making $100m+ each rather than hundreds making ~$1m each.

Too bad those experts are not worth their 300 million packages. I've seen the google scholars of the confirmed crazy comp hires and it's not Yann Lecun tier that's for sure.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#294

Post by TheBicPen »

moffkalast wrote: post #47
a-dub wrote: post #33

Earlier quoted context omitted.

"open source" means there should be a script that downloads all the training materials and then spins up a pipeline that trains end to end. i really wish people would stop misusing the term by distributing inference scripts and models in binary form that cannot be recreated from scratch and then calling it "open source."

I'd agree but we're beyond hopelessly idealistic. That sort of approach only helps your competition who will use it to build a closed product and doesn't give anything of worth to people who want to actually use the model because they have no means to train it. Hell most people can barely scrape up enough hardware to even run inference. Reproducing models is also not very ecological in when it comes down to it, do we…

> I'd agree but we're beyond hopelessly idealistic. That sort of approach only helps your competition who will use it to build a closed product

That same argument can be applied to open-source (non-model) software, and is about as true there. It comes down to the business model. If anything, crating a closed-sourced copy of a piece of FOSS software is easier than an AI model since running a compiler doesn't cost millions of dollars.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#295

Post by Palmik »

baalimago wrote: post #284

Unfortunate how many of the 'non mainstream' models are poor at function handling. I'm trying K2 out via Novia AI and it consistently fails to format function calls, breaking the reasoning flow.

This is most likely issue on the side of the inference provider: https://github.com/MoonshotAI/K2-Vendor-Verifier

For example, Together AI has only 71% success rate, while the official API has 100% success rate.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#296

Post by Gigachad »

toephu2 wrote: post #257
the_mitsuhiko wrote: post #51

Earlier quoted context omitted.

And Europeans don't it because quite frankly, we're not really doing anything particularly impressive with AI sadly.

Europe gave us cookie popups on every single website.

Only ones with invasive spyware cookies. Essential site function cookies do not require a consent banner.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#297

Post by Palmik »

gradus_ad wrote: post #265

While I absolutely support these open source models, there is an interesting angle to consider... If I were a Chinese partisan looking to inflict a devastating blow to the US, taking the AI hype wind out of American tech valuation sails would seem a great option. How best to do this? Release highly performant models... For free! Extremely efficient in terms of RMB spent vs (unrealized) USD lost. But surely, these mod…

On the other hand, several startups such as Cursor and Cognition+Windsurf are building their new models on top of the open source Chinese models.

Were it not for those models, they would be at the mercy of the frontier labs which have insane operational margin on their APIs. As a result you'd see much more consolidation.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#298

Post by vanviegen »

Der_Einzige wrote: post #291
simonw wrote: post #288

Earlier quoted context omitted.

Hard to be sure because the source of that information isn't known, but generally when people talk about training costs like this they include more than just the electricity but exclude staffing costs. Other reported training costs tend to include rental of the cloud hardware (or equivalent if the hardware is owned by the company), e.g. NVIDIA H100s are sometimes priced out in cost-per-hour.

Citation needed on "generally when people talk about training costs like this they include more than just the electricity but exclude staffing costs". It would be simply wrong to exclude the staffing costs. When each engineer costs well over 1 million USD in total costs year over year, you sure as hell account for them.

No, because what people are generally trying to express with numbers like these, is how much compute went into training. Perhaps another measure, like zettaflop or something would have made more sense.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#299

Post by pu_pe »

jjice wrote: post #152
pu_pe wrote: post #42

Four independent Chinese companies released extremely good open source models in the past few months (DeepSeek, Qwen/Alibaba, Kimi/Moonshot, GLM/Z.ai). No American or European companies are doing that, including titans like Meta. What gives?

I get what you mean, but OpenAI did release the gpt-oss in August, just three months ago. I've had a very good experience with those models. https://openai.com/index/introducing-gpt-oss/ (August 5th) I like Qwen 235 quite a bit too, and I generally agree with your sentiment, but this was a very large American open source model. Unless we're getting into the complications on what "open source" model actually means, in…

You're totally right. Ironically I am using gpt-oss for a project right now, I think its quality is comparable to the ones I mentioned.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#300

Post by kachapopopow »

ElijahLynn wrote: post #228

Why is the 4bit version 1.2TB and the non-4bit version 650GB? https://huggingface.co/mlx-community/Kimi-K2-Thinking-4bit https://huggingface.co/mlx-community/Kimi-K2-Thinking

I think it the default version here might be 2.5bit or something
Post reply on HN