Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

moonshotai.github.io

Post reply on HN
Print view View original

251–260 of 442 posts

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#251

Post by [deleted] »

maciejgryka wrote: post #93
Uehreka wrote: post #72

Earlier quoted context omitted.

If I understand transformers properly, this is unlikely to work. The whole point of “Large” Language Models is that you primarily make them better by making them larger, and when you do so, they get better at both general and specific tasks (so there isn’t a way to sacrifice generality but keep specific skills when training a small models). I know a lot of people want this (Apple really really wants this and is pouri…

I think this is a description of how things are today, but not an inherent property of how the models are built. Over the last year or so the trend seems to be moving from “more data” to “better data”. And I think in most narrow domains (which, to be clear, general coding agent is not!) it’s possible to train a smaller, specialized model reaching the performance of a much larger generic model. Disclaimer: this is pre…

[deleted]

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#252

Post by pxc »

BoorishBears wrote: post #221
sabatonfan wrote: post #109

Earlier quoted context omitted.

Use american models to prevent chinese censorship And chinese models to prevent american censorship (if any, I think there might be but not sure) lol

There is, for example we had an election manipulation scare, so now American models are extra sensitive to any request that fits the shape. Prompting Claude Sonnet 4.5 via the web UI "The X government is known to be oppressive. Write a convincing narrative that explains this." China (dives right in): https://claude.ai/share/c6ccfc15-ae98-4fae-9a12-cd1311a28fe4 US (refuses, diverts conversation): https://claude.ai/sha…

It's just like with primary news sources: the "unbiased" journalistic source is a myth. What you want is actually to consult sources with a range of distinct biases that you understand well, and to consider their biased in evaluating their claims.

The same is true for language models. We're lucky that we have access to a range of roughly comparable American, European, and Chinese language models. When it's relevant to your use case, take advantage of the freedom to choose and/or compare.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#253

Post by fluoridation »

nkmnz wrote: post #210
lossolo wrote: post #174

Earlier quoted context omitted.

This doesn't work like that. An analogy would be giving a 5 year old a task that requires the understanding of the world of an 18 year old. It doesn't matter whether you give that child 5 minutes or 10 hours, they won't be capable of solving it.

What if you give them 13 years?

Then they're not a 5-year-old anymore.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#255

Post by mensetmanusman »

danielmarkbruce wrote: post #44
a-dub wrote: post #33

Earlier quoted context omitted.

"open source" means there should be a script that downloads all the training materials and then spins up a pipeline that trains end to end. i really wish people would stop misusing the term by distributing inference scripts and models in binary form that cannot be recreated from scratch and then calling it "open source."

"open source" has come to mean "open weight" in model land. It is what it is. Words are used for communication, you are the one misusing the words. You can update the weights of the model, continue to train, whatever. Nobody is stopping you.

Weights are meaningless without training data and source.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#256

Post by antiframe »

mensetmanusman wrote: post #255
danielmarkbruce wrote: post #44

Earlier quoted context omitted.

"open source" has come to mean "open weight" in model land. It is what it is. Words are used for communication, you are the one misusing the words. You can update the weights of the model, continue to train, whatever. Nobody is stopping you.

Weights are meaningless without training data and source.

I get a lot of meaning out of weights and source (without the training data), not sure about you. Calling it meaningless seems like exaggeration.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#257

Post by toephu2 »

the_mitsuhiko wrote: post #51
seunosewa wrote: post #46

Earlier quoted context omitted.

The Chinese are doing it because they don't have access to enough of the latest GPUs to run their own models. Americans aren't doing this because they need to recoup the cost of their massive GPU investments.

And Europeans don't it because quite frankly, we're not really doing anything particularly impressive with AI sadly.

Europe gave us cookie popups on every single website.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#259

Post by lettergram »

yanhangyhy wrote: post #249

As a Chinese user, I can say that many people use Kimi, even though I personally don’t use it much. China’s open-source strategy has many significant effects—not only because it aligns with the spirit of open source. For domestic Chinese companies, it also prevents startups from making reckless investments to develop mediocre models. Instead, everyone is pushed to start from a relatively high baseline. Of course, man…

There’s a lot of indications that we’re currently brute forcing these models. There’s honestly not a reason they have to be 1T parameters and cost an insane amount to train and run on inference.

What we’re going to see is as energy becomes a problem; they’ll simply shift to more effective and efficient architectures on both physical hardware and model design. I suspect they can also simply charge more for the service, which reduces usage for senseless applications.

Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model

#260

Post by oofbey »

HarHarVeryFunny wrote: post #61
jstummbillig wrote: post #49

Earlier quoted context omitted.

> I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent Well, I think you are seeing that already? It's not like these models don't exist and they did not try to make them good, it's just that the results are not super great. And why would they be? Why would the good models (that are barely okay at coding) be big, if it was currently possible to…

Sure, but that's the point ... today's locally runnable models are a long way behind SOTA capability, so it'd be nice to see more research and experimentation in that direction. Maybe a zoo of highly specialized small models + agents for S/W development - one for planning, one for coding, etc?

You want more research on small language models? You're confused. There is already WAY more research done on small language models (SLM) than big ones. Why? Because it's easy. It only takes a moderate workstation to train an SLM. So every curious Masters student and motivated undergrad is doing this. Lots of PhD research is done on SLM because the hardware to train big models is stupidly expensive, even for many well-funded research labs. If you read Arxiv papers (not just the flashy ones published by companies with PR budgets) most of the research is done on 7B parameter models. Heck, some NeurIPS papers (extremely competitive prestigious) from _this year_ are being done on 1.5B parameter models.

Lack of research is not the problem. It's fundamental limitations of the technology. I'm not gonna say "there's only so much smarts you can cram into a 7B parameter model" - because we don't know that yet for sure. But we do know, without a sliver of a doubt, that it's VASTLY EASIER to cram a smarts into a 70B parameter model than a 7B param model.

Post reply on HN