Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

moonshotai.github.io

Post reply on HN
Print view View original

161–170 of 442 posts

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#161

Post by utopiah »

the_mitsuhiko wrote: post #51
seunosewa wrote: post #46

Earlier quoted context omitted.

The Chinese are doing it because they don't have access to enough of the latest GPUs to run their own models. Americans aren't doing this because they need to recoup the cost of their massive GPU investments.

And Europeans don't it because quite frankly, we're not really doing anything particularly impressive with AI sadly.

> we're not really doing anything particularly impressive with AI sadly.

Well, that's true... but also nobody else is. Making something popular isn't particularly impressive.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#162

Post by raincole »

pu_pe wrote: post #42

Four independent Chinese companies released extremely good open source models in the past few months (DeepSeek, Qwen/Alibaba, Kimi/Moonshot, GLM/Z.ai). No American or European companies are doing that, including titans like Meta. What gives?

Do you think which one has the higher market share:

1) The four models you mentioned, combined

or

2) ChatGPT

?

What gives? Because if people are willing to pay you, you don't say "ok I don't want your money I'll provide my service for free."

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#164

Post by yorwba »

Aurornis wrote: post #110
cantor_S_drug wrote: post #73

Earlier quoted context omitted.

In CS algorithms, we have space vs time tradeoffs. In LLMs, we will have bigger weights vs test-time compute tradeoffs. A smaller model can get "there" but it will take longer.

> In LLMs, we will have bigger weights vs test-time compute tradeoffs. A smaller model can get "there" but it will take longer. Assuming both are SOTA, a smaller model can't produce the same results as a larger model by giving it infinite time. Larger models inherently have more room for training more information into the model. No amount of test-retry cycle can overcome all of those limits. The smaller models will j…

It's true that to train more information into the model you need more trainable parameters, but when people ask for small models, they usually mean models that run at acceptable speeds on their hardware. Techniques like mixture-of-experts allow increasing the number of trainable parameters without requiring more FLOPs, so they're large in one sense but small in another.

And you don't necessarily need to train all information into the model, you can also use tool calls to inject it into the context. A small model that can make lots of tool calls and process the resulting large context could obtain the same answer that a larger model would pull directly out of its weights.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#165

Post by twotwotwo »

HarHarVeryFunny wrote: post #29

It's good to see more competition, and open source, but I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent as opposed to a trillion parameter one. The ideal case would be something that can be run locally, or at least on a modest/inexpensive cluster. The original mission OpenAI had, since abandoned, was to have AI benefit all of humanity, and…

I think there is a lot of progress on efficient useful models recently.

I've seen GLM-4.6 getting mention for good coding results from a model that's much smaller than Kimi (~350b params) and seen it speculated that Windsurf based their new model on it.

This Kimi release is natively INT4, with quantization-aware training. If that works--if you can get really good results from four-bit parameters--it seems like a really useful tool for any model creator wanting efficient inference.

DeepSeek's v3.2-Exp uses their sparse attention technique to make longer-context training and inference more efficient. Its output's being priced at 60% less than v3.1 (though that's an imperfect indicator of efficiency). They've also quietly made 'thinking' mode need fewer tokens since R1, helping cost and latency.

And though it's on the proprietary side, Haiku 4.5 approaching Sonnet 4 coding capability (at least on benches Anthropic released) also suggests legitimately useful models can be much smaller than the big ones.

There's not yet a model at the level of any of the above that's practical for many people to run locally, though I think "efficient to run + open so competing inference providers can run it" is real progress.

More important it seems like there's a good trendline towards efficiency, and a bunch of techniques are being researched and tested that, when used together, could make for efficient higher-quality models.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#166

Post by prmph »

Aurornis wrote: post #105
HarHarVeryFunny wrote: post #61

Earlier quoted context omitted.

Sure, but that's the point ... today's locally runnable models are a long way behind SOTA capability, so it'd be nice to see more research and experimentation in that direction. Maybe a zoo of highly specialized small models + agents for S/W development - one for planning, one for coding, etc?

> today's locally runnable models are a long way behind SOTA capability SOTA models are larger than what can be run locally, though. Obviously we'd all like to see smaller models perform better, but there's no reason to believe that there's a hidden secret to making small, locally-runnable models perform at the same level as Claude and OpenAI SOTA models. If there was, Anthropic and OpenAI would be doing it. There's…

You're still missing the point. The comment you're responding to is talking about specialized models

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#167

Post by pphysch »

raincole wrote: post #162
pu_pe wrote: post #42

Four independent Chinese companies released extremely good open source models in the past few months (DeepSeek, Qwen/Alibaba, Kimi/Moonshot, GLM/Z.ai). No American or European companies are doing that, including titans like Meta. What gives?

Do you think which one has the higher market share: 1) The four models you mentioned, combined or 2) ChatGPT ? What gives? Because if people are willing to pay you, you don't say "ok I don't want your money I'll provide my service for free."

Open-weight (Chinese) models have infinitely more market share in domains where giving your data to OpenAI is not acceptable

Like research labs and so on. Even at US universities

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#168

Post by nodja »

NiloCK wrote: post #129

Maybe a dumb question but: what is a "reasoning model"? I think I get that "reasoning" in this context refers to dynamically budgeting scratchpad tokens that aren't intended as the main response body. But can't any model do that, and it's just part of the system prompt, or more generally, the conversation scaffold that is being written to. Or does a "reasoning model" specifically refer to models whose "post training"…

Any model that does thinking inside style tokens before it answers.

This can be done with finetuning/RL using an existing pre-formatted dataset, or format based RL where the model is rewarded for both answering correct and using the right format.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#169

Post by walthamstow »

bigyabai wrote: post #142
NiloCK wrote: post #139

Earlier quoted context omitted.

I'd expect the same (fine tuning to be better than mere prompting) for most anything. So a model is or is not "a reasoning model" according to the extent of a fine tune. Are there specific benchmarks that compare models vs themselves with and without scratchpads? High with:without ratios being reasonier models? Curious also how much a generalist model's one-shot responses degrade with reasoning post-training.

> Are there specific benchmarks that compare models vs themselves with and without scratchpads? Yep, it's pretty common for many models to release an instruction-tuned and thinking-tuned model and then bench them against each other. For instance, if you scroll down to "Pure text performance" there's a comparison of these two Qwen models' performance: https://huggingface.co/Qwen/Qwen3-VL-30B-A3B-Thinking

Thanks for the Qwen tip. Interesting how much of a difference reasoning makes for coding.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#170

Post by Gracana »

unleaded wrote: post #86
Gracana wrote: post #74

Earlier quoted context omitted.

Interesting, I have the opposite impression. I want to like it because it's the biggest model I can run at home, but its punchy style and insistence on heavily structured output scream "tryhard AI." I was really hoping that this model would deviate from what I was seeing in their previous release.

what do you mean by "heavily structured output"? i find it generates the most natural-sounding output of any of the LLMs—cuts straight to the answer with natural sounding prose (except when sometimes it decides to use chat-gpt style output with its emoji headings for no reason). I've only used it on kimi.com though, wondering what you're seeing.

Yeah, by "structured" I mean how it wants to do ChatGPT-style output with headings and emoji and lists and stuff. And the punchy style of K2 0905 as shown in the fiction example in the linked article is what I really dislike. K2 Thinking's output in that example seems a lot more natural.

I'd be totally on board if cut straight to the answer with natural sounding prose, as you described, but for whatever reason that has not been my experience.

Post reply on HN