When I open this page, all I see is a word pad like text area with buttons on top and sample text inside. Don't see anything about any llm. I am on phone. Page is being opened via embedded view in an HN client.
Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
181–190 of 442 posts
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#182How does one effectively use something like this locally with consumer-grade hardware?
Consumer-grade hardware? Even at 4bits per param you would need 500GB of GPU VRAM just to load the weights. You also need VRAM for KV cache.
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#183Earlier quoted context omitted.
Why is this a benchmark though? It doesn’t correlate with intelligence
It started as a joke, but over time performance on this one weirdly appears to correlate to how good the models are generally. I'm not entirely sure why!
it has to do with world model perception. these models don't have it but some can approximate it better than others.
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#184It's good to see more competition, and open source, but I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent as opposed to a trillion parameter one. The ideal case would be something that can be run locally, or at least on a modest/inexpensive cluster. The original mission OpenAI had, since abandoned, was to have AI benefit all of humanity, and…
If NVIDIA had any competition we'd be able to run these larger models at home by now instead of being saddled with these 16GB midgets.
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#185Earlier quoted context omitted.
And Europeans don't it because quite frankly, we're not really doing anything particularly impressive with AI sadly.
Honestly, do we need to? If the Chinese release SOTA open source models, why should we invest a ton just to have another one? We can just use theirs, that's the beauty of open source.
For the vast majority, they're not "open source" they're "open weights". They don't release the training data or training code / configs.
It's kind of like releasing a 3d scene rendered to a JPG vs actually providing someone with the assets.
You can still use it, and it's possible to fine-tune it, but it's not really the same. There's tremendous soft power in deciding LLM alignment and material emphasis. As these things become more incorporated into education, for instance, the ability to frame "we don't talk about ba sing se" issues are going to be tremendously powerful.
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#186Earlier quoted context omitted.
> today's locally runnable models are a long way behind SOTA capability SOTA models are larger than what can be run locally, though. Obviously we'd all like to see smaller models perform better, but there's no reason to believe that there's a hidden secret to making small, locally-runnable models perform at the same level as Claude and OpenAI SOTA models. If there was, Anthropic and OpenAI would be doing it. There's…
You're still missing the point. The comment you're responding to is talking about specialized models
The point is still valid. If the big companies could save money running multiple small specialised models on cheap hardware, they wouldn't be spending billions on the highest spec GPUs.
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#187Earlier quoted context omitted.
When the machines become depressed and anxious we'll know they've achieved true intelligence. This is only partly a joke.
This already happens! There have been many reports of CLI AI tools getting frustrated, giving up, and just deleting the whole codebase in anger.
There are many reports of CLI AI tools displaying words that humans express when they are frustrated and about to give up. Just what they have been trained on. That does not mean they have emotions. And "deleting the whole codebase" sounds more interesting, but I assume is the same thing. "Frustrated" words lead to frustrated actions. Does not mean the LLM was frustrated. Just that in its training data those things happened so it copied them in that situation.
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#188Earlier quoted context omitted.
This already happens! There have been many reports of CLI AI tools getting frustrated, giving up, and just deleting the whole codebase in anger.
There are many reports of CLI AI tools displaying words that humans express when they are frustrated and about to give up. Just what they have been trained on. That does not mean they have emotions. And "deleting the whole codebase" sounds more interesting, but I assume is the same thing. "Frustrated" words lead to frustrated actions. Does not mean the LLM was frustrated. Just that in its training data those things h…
This is a fundamental philosophical issue with no clear resolution.
The same argument could be made about people, animals, etc...
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#189Earlier quoted context omitted.
In CS algorithms, we have space vs time tradeoffs. In LLMs, we will have bigger weights vs test-time compute tradeoffs. A smaller model can get "there" but it will take longer.
This doesn't work like that. An analogy would be giving a 5 year old a task that requires the understanding of the world of an 18 year old. It doesn't matter whether you give that child 5 minutes or 10 hours, they won't be capable of solving it.
I think the question of what can be achieved with a small model comes down to what needs knowledge vs what needs experience. A small model can use tools like RAG if it is just missing knowledge, but it seems hard to avoid training/parameters where experience is needed - knowing how to perceive then act.
There is obviously also some amount (maybe a lot) of core knowledge and capability needed even to be able to ask the right questions and utilize the answers.
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#190uv tool install llm llm install llm-moonshot llm keys set moonshot # paste key llm -m moonshot/kimi-k2-thinking 'Generate an SVG of a pelican riding a bicycle' https://tools.simonwillison.net/svg-render#%3Csvg%20width%3D... Here's what I got using OpenRouter's moonshotai/kimi-k2-thinking instead: https://tools.simonwillison.net/svg-render#%20%20%20%20%3Csv...
Does the run pin the temperature to 0 for consistency?
No, I've never tried that.