In the mean time, Sam is looking at putting more servers on the moon.
Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
91–100 of 442 posts
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#92Weird. I just tried it and it fails when I ask: "Tell me about the 1989 Tiananmen Square massacre".
yes yes Chinese models have Chinese censorship, we don't need to belabour this point every time
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#93Earlier quoted context omitted.
Sure, but that's the point ... today's locally runnable models are a long way behind SOTA capability, so it'd be nice to see more research and experimentation in that direction. Maybe a zoo of highly specialized small models + agents for S/W development - one for planning, one for coding, etc?
If I understand transformers properly, this is unlikely to work. The whole point of “Large” Language Models is that you primarily make them better by making them larger, and when you do so, they get better at both general and specific tasks (so there isn’t a way to sacrifice generality but keep specific skills when training a small models). I know a lot of people want this (Apple really really wants this and is pouri…
I think this is a description of how things are today, but not an inherent property of how the models are built. Over the last year or so the trend seems to be moving from “more data” to “better data”. And I think in most narrow domains (which, to be clear, general coding agent is not!) it’s possible to train a smaller, specialized model reaching the performance of a much larger generic model.
Disclaimer: this is pretty much the thesis of a company I work for, distillabs.ai but other people say similar things e.g. https://research.nvidia.com/labs/lpr/slm-agents/
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#94So Apple is about to pay OpenAI 1 B usd pr year for what moonshot is giving for free?
You haven't seen Gemini 3 yet. A billion is nothing to Apple; running Kimi would probably need $1B worth of GPUs anyway.
People don't get that Apple would need an enormous data center buildout to provide a good AI experience on their millions of deployed devices. Google is in the exascale datacenter buildout business, while Apple isn't.
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#95It's good to see more competition, and open source, but I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent as opposed to a trillion parameter one. The ideal case would be something that can be run locally, or at least on a modest/inexpensive cluster. The original mission OpenAI had, since abandoned, was to have AI benefit all of humanity, and…
I think it’s going to be a while before we see small models (defined roughly as “runnable on reasonable consumer hardware”) do a good job at general coding tasks. It’s a very broad area! You can do some specific tasks reasonably well (eg I distilled a toy git helper you can run locally here https://github.com/distil-labs/gitara), but “coding” is such a big thing that you really need a lot of knowledge to do it well.
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#96Four independent Chinese companies released extremely good open source models in the past few months (DeepSeek, Qwen/Alibaba, Kimi/Moonshot, GLM/Z.ai). No American or European companies are doing that, including titans like Meta. What gives?
The answer is simply that no one would pay to use them for a number of reasons including privacy. They have to give them away and put up some semblance of openness. No option really.
I don’t think there’s any privacy that OpenAI or Anthropic are giving you that DeepSeek isn’t giving you. ChatGPT usage logs were held by court order at one point.
It’s true that DeepSeek won’t give you reliable info on Tiananmen Square but I would argue that’s a very rare use case in practice. Most people will be writing boilerplate code or summarizing mundane emails.
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#97Earlier quoted context omitted.
> I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent Well, I think you are seeing that already? It's not like these models don't exist and they did not try to make them good, it's just that the results are not super great. And why would they be? Why would the good models (that are barely okay at coding) be big, if it was currently possible to…
In CS algorithms, we have space vs time tradeoffs. In LLMs, we will have bigger weights vs test-time compute tradeoffs. A smaller model can get "there" but it will take longer.
I have spent the last 2.5 years living like a monk to maintain an app across all paid LLM providers and llama.cpp.
I wish this was true.
It isn't.
"In algorithms, we have space vs time tradeoffs, therefore a small LLM can get there with more time" is the same sort of "not even wrong" we all smile about us HNers doing when we try applying SWE-thought to subjects that aren't CS.
What you're suggesting amounts to "monkeys on typewriters will write entire works of Shakespeare eventually" - neither in practice, nor in theory, is this a technical claim, or something observable, or even stood up as a one-off misleading demo once.
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#98I am sure they cherry-picked the examples but still, wow. Having spent a considerable amount of time trying to introduce OSS models in my workflows I am fully aware of their short comings. Even frontier models would struggle with such outputs (unless you lead the way, help break down things and maybe even use sub-agents). Very impressed with the progress. Keeps me excited about what’s to come next!
Subjectively I find Kimi is far "smarter" than the benchmarks imply, maybe because they game then less than US labs
My impression as well!
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#99Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#100It's good to see more competition, and open source, but I'd be much more excited to see what level of coding and reasoning performance can be wrung out of a much smaller LLM + agent as opposed to a trillion parameter one. The ideal case would be something that can be run locally, or at least on a modest/inexpensive cluster. The original mission OpenAI had, since abandoned, was to have AI benefit all of humanity, and…
I don't understand. We already have that capability in our skulls. It's also "already there", so it would be a waste to not use it.
Software development is one of the areas where LLMs really are useful, whether that's vibe coding disposable software, or more structured use for serious development.
I've been a developer for 40+ years, and very good at it, but for some tasks it's not about experience or overcoming complexity - just a bunch of grunt work that needs to come together. The other day I vibe coded a prototype app, just for one-time demo use, in less than 15 min that probably would have taken a week to write by hand, assuming one was already familiar with the tech stack.
Developing is fun, and a brain is a terrible thing to waste, but today not using LLMs where appropriate for coding doesn't make any sense if you value your time whatsoever.