Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

moonshotai.github.io

Post reply on HN
Print view View original

201–210 of 442 posts

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#201

Post by hellzbellz123 »

jiggawatts wrote: post #188
lukan wrote: post #187

Earlier quoted context omitted.

There are many reports of CLI AI tools displaying words that humans express when they are frustrated and about to give up. Just what they have been trained on. That does not mean they have emotions. And "deleting the whole codebase" sounds more interesting, but I assume is the same thing. "Frustrated" words lead to frustrated actions. Does not mean the LLM was frustrated. Just that in its training data those things h…

This is a fundamental philosophical issue with no clear resolution. The same argument could be made about people, animals, etc...

This only seems to be an issue for wishy washy types that insist gpt is alive.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#202

Post by root_axis »

0xjmp wrote: post #130
barrell wrote: post #114

Earlier quoted context omitted.

I believe they mean distribution (inference). The Chinese model is currently B.Y.O.GPU. The American model is GPUaaS

Why is inference less attainable when it technically requires less GPU processing to run? Kimi has a chat app on their page using K2 so they must have figured out inference to some extent.

> it technically requires less GPU processing to run

Not when you have to scale. There's a reason why every LLM SaaS aggressively rate limits and even then still experiences regular outages.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#203

Post by wmf »

btian wrote: post #195
criley2 wrote: post #101

Earlier quoted context omitted.

Apple is buying a model from Google, not inference. Apple will host the model themselves. It's very simple: Apple absolutely refuses to send all their user data to Google.

Then why did Apple have a $20B a year search deal with Google?

The argument can be made that when people search Google they know they are using Google but when they use Siri they assume that their data is not going to Google. I think this is more likely to be solved contractually than having Gemini running on a datacenter full of M5 Ultra servers.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#204

Post by raincole »

pphysch wrote: post #167
raincole wrote: post #162

Earlier quoted context omitted.

Do you think which one has the higher market share: 1) The four models you mentioned, combined or 2) ChatGPT ? What gives? Because if people are willing to pay you, you don't say "ok I don't want your money I'll provide my service for free."

Open-weight (Chinese) models have infinitely more market share in domains where giving your data to OpenAI is not acceptable Like research labs and so on. Even at US universities

Cool, and? If these models were hosted in China, the labs you mentioned wouldn't be paying them, right?

Now you have the answer to "what gives" above.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#205

Post by EMM_386 »

benjiro wrote: post #178
HarHarVeryFunny wrote: post #29

It's good to see more competition, and open source, but I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent as opposed to a trillion parameter one. The ideal case would be something that can be run locally, or at least on a modest/inexpensive cluster. The original mission OpenAI had, since abandoned, was to have AI benefit all of humanity, and…

What i do not understand is why we are not seeing specialized models that go down to single experts. I do not need models that know how to program in Python, Rust, ... when i only use Go and Html. So we are we not seeing models that have very specialized experts, where for instance: * General interpreter model, that holds context/memory * Go Model * Html model if there is space in memory. * SQL model if there is spac…

I think one of the issues is that LLMs can't have a "Go" model and an "HTML model". I mean, they can but what would that contain? It's not the language-specific features that make models large.

When models work on your code base, they do not "see" things like this, which is why they can go through an entire code base with variable names they have never seen before, function signatures they have never seen before, and directory structures that have never seen before and not have a problem.

You need that "this is a variable, which is being passed to a function which recursively does ..." part. This is not something language specific, it's the high level understanding of how languages and systems operate. A variable is a variable whether in JavaScript or C++ and LLMs can "see" it as such. The details are different but it's that layer of "this is a software interface", "this is a function pointer" is outside of the "Go" or "Python" or "C#" model.

I don't know how large the main model would have to be vs. the specialized models in order to pick this dynamic up.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#206

Post by jychang »

0xjmp wrote: post #130
barrell wrote: post #114

Earlier quoted context omitted.

I believe they mean distribution (inference). The Chinese model is currently B.Y.O.GPU. The American model is GPUaaS

Why is inference less attainable when it technically requires less GPU processing to run? Kimi has a chat app on their page using K2 so they must have figured out inference to some extent.

That entirely depends on the number of users.

Inference is usually less gpu-compute heavy, but much more gpu-vram heavy pound-for-pound compared to training. General rule of thumb is that you need 20x more vram for training a model with X params, than for inference for that same size model. So assuming batch size b, then serving more than 20*b users would tilt vram use on the side of inference.

This isn't really accurate; it's an extremely rough rule of thumb and ignores a lot of stuff. But it's important to point out that inference is quickly adding to costs for all AI companies. Deepseek claims that they used $5.6mil to train Deepseek R1; that's about 10-20 trillion tokens at their current pricing- or 1 million users sending just 100 requests at full context size.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#207

Post by andai »

HarHarVeryFunny wrote: post #29

It's good to see more competition, and open source, but I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent as opposed to a trillion parameter one. The ideal case would be something that can be run locally, or at least on a modest/inexpensive cluster. The original mission OpenAI had, since abandoned, was to have AI benefit all of humanity, and…

I used to be obsessed with what's the smartest LLM, until I tried actually using them for some tasks and realized that the smaller models did the same task way faster.

So I switched my focus from "what's the smartest model" to "what's the smallest one that can do my task?"

With that lens, "scores high on general intelligence benchmarks" actually becomes a measure of how overqualified the model is, and how much time, money and energy you are wasting.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#208

Post by andai »

cantor_S_drug wrote: post #73
jstummbillig wrote: post #49

Earlier quoted context omitted.

> I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent Well, I think you are seeing that already? It's not like these models don't exist and they did not try to make them good, it's just that the results are not super great. And why would they be? Why would the good models (that are barely okay at coding) be big, if it was currently possible to…

In CS algorithms, we have space vs time tradeoffs. In LLMs, we will have bigger weights vs test-time compute tradeoffs. A smaller model can get "there" but it will take longer.

Actually it depends on the task. For many tasks, a smaller model can handle it, and it gets there faster!

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#209

Post by andai »

a-dub wrote: post #33
HarHarVeryFunny wrote: post #29

It's good to see more competition, and open source, but I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent as opposed to a trillion parameter one. The ideal case would be something that can be run locally, or at least on a modest/inexpensive cluster. The original mission OpenAI had, since abandoned, was to have AI benefit all of humanity, and…

"open source" means there should be a script that downloads all the training materials and then spins up a pipeline that trains end to end. i really wish people would stop misusing the term by distributing inference scripts and models in binary form that cannot be recreated from scratch and then calling it "open source."

The meaning of Open Source

1990: Free Software

2000: Open Source: Finally we sanitized ourselves of that activism! It was scaring away customers!

2010: Source is available (under our very restrictive license)

2020: What source?

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#210

Post by nkmnz »

lossolo wrote: post #174
cantor_S_drug wrote: post #73

Earlier quoted context omitted.

In CS algorithms, we have space vs time tradeoffs. In LLMs, we will have bigger weights vs test-time compute tradeoffs. A smaller model can get "there" but it will take longer.

This doesn't work like that. An analogy would be giving a 5 year old a task that requires the understanding of the world of an 18 year old. It doesn't matter whether you give that child 5 minutes or 10 hours, they won't be capable of solving it.

What if you give them 13 years?
Post reply on HN