Any word on what it takes to run this thing?
Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
191–200 of 442 posts
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#192Is the price here correct? https://openrouter.ai/moonshotai/kimi-k2-thinking Would be $0,60 for input and $2,50 for 1 million output tokens. If the model is really that good it's 4x cheaper than comparable models. It's hosted at a loss or the others have a huge margin? I might miss something here. Would love some expert opinion :) FYI: the non thinking variant has the same price.
In short, the others have a huge margin if you ignore training costs. See https://martinalderson.com/posts/are-openai-and-anthropic-re... for details.
Somehow that article totally ignored the insane pricing of cached input tokens set by Anthropic and OpenAI. For agentic coding, typically 90~95% of the inference cost is attributed to cached input tokens, and a scrappy China company can do it almost for free: https://api-docs.deepseek.com/news/news0802
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#193Earlier quoted context omitted.
Sure, but that's the point ... today's locally runnable models are a long way behind SOTA capability, so it'd be nice to see more research and experimentation in that direction. Maybe a zoo of highly specialized small models + agents for S/W development - one for planning, one for coding, etc?
If I understand transformers properly, this is unlikely to work. The whole point of “Large” Language Models is that you primarily make them better by making them larger, and when you do so, they get better at both general and specific tasks (so there isn’t a way to sacrifice generality but keep specific skills when training a small models). I know a lot of people want this (Apple really really wants this and is pouri…
The issue isn't even 'quality' per se (for many tasks a small model would do fine), its for "agentic" workflows it _quickly_ runs out of context. Even 32GB VRAM is really very limiting.
And when I mean agentic, i mean something even like this - 'book a table from my emails', which involves looking at 5k+ tokens of emails, 5k tokens of search results, then confirming with the user etc. It's just not feasible on most hardware right now - even if the models are 1-2GB, you'll burn thru the rest in context so quickly.
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#194Earlier quoted context omitted.
There are many reports of CLI AI tools displaying words that humans express when they are frustrated and about to give up. Just what they have been trained on. That does not mean they have emotions. And "deleting the whole codebase" sounds more interesting, but I assume is the same thing. "Frustrated" words lead to frustrated actions. Does not mean the LLM was frustrated. Just that in its training data those things h…
This is a fundamental philosophical issue with no clear resolution. The same argument could be made about people, animals, etc...
The difference is, people and animals have a body, nerve system and in general those mushy things we think are responsible for emotions.
Computers don't have any of that. And LLM's in particular neither. They were trained to simulate human text responses, that's all. How to get from there to emotions - where is the connection?
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#195Earlier quoted context omitted.
People don't get that Apple would need an enormous data center buildout to provide a good AI experience on their millions of deployed devices. Google is in the exascale datacenter buildout business, while Apple isn't.
Apple is buying a model from Google, not inference. Apple will host the model themselves. It's very simple: Apple absolutely refuses to send all their user data to Google.
Then why did Apple have a $20B a year search deal with Google?
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#196Earlier quoted context omitted.
In CS algorithms, we have space vs time tradeoffs. In LLMs, we will have bigger weights vs test-time compute tradeoffs. A smaller model can get "there" but it will take longer.
This doesn't work like that. An analogy would be giving a 5 year old a task that requires the understanding of the world of an 18 year old. It doesn't matter whether you give that child 5 minutes or 10 hours, they won't be capable of solving it.
but in 13 years, will they be capable?
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#197Earlier quoted context omitted.
Love seeing this benchmark become more iconic with each new model release. Still in disbelief at the GPT-5 variants' performance in comparison but its cool to see the new open source models get more ambitious with their attempts.
Only until they start incorporating this test into their training data.
Dataset contamination alone won't get them good-looking SVG pelicans on bicycles though, they'll have to either cheat this particular question specifically or train it to make vector illustrations in general. At which point it can be easily swapped for another problem that wasn't in the data.
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#198Earlier quoted context omitted.
Why is this a benchmark though? It doesn’t correlate with intelligence
What test would be better correlated with intelligence and why?
A mathematical exam problem not in the training set because mathematical and logical reasoning are usually what people mean by intelligence.
I don’t think Einstein or von Neumann could do this SVG problem, does that mean they’re dumb?
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#199TLDR; this is an alibaba funded start-up out of Beijing
Okay, I'm sorry but I have to say wtf named this thing. Moonshot AI is such an overused generic name that I had to ask an LLM which company this is. This is just Alibaba hedging their Qwen model.
This company is far from "open source", it's had over $1B USD in funding.
Re: Kimi K2 Thinking, a SOTA open-source trillion-parameter reasoning model
#200Available on OpenRouter already as well in case anyone wants to try it there: https://openrouter.ai/moonshotai/kimi-k2-thinking
laggy as all hell