[deleted]
The Kimi K3 Moment
41–50 of 644 posts
Re: The Kimi K3 Moment
#42I think the biggest problem with Chinese models is that they seems to overthink for most of the tasks, especially for smaller ones. The OpenAI models have in my experience only gotten better in terms of efficiency.
Yes, this (imo) is a clear result of benchmaxxing. You can get a much better score on most "intelligence" benchmarks by massively over-saturating reasoning. This looks good on those, but for actual daily usage makes the models much less effective: I don't want a model I use for coding to burn a bunch of reasoning (read: time) on trivial tasks.
I strongly suspect the flip side is that in the future it enables you to train smarter models by "distilling" the end result of the super duper heavily thinking models.
Re: The Kimi K3 Moment
#43I tried Kimi K3 on a task I've done with every other model I use regularly ( https://swelljoe.com/post/i-let-every-agent-implement-its-ow... ) and found it chewed a lot longer on the problem and ate up almost the entirety of a 5 hour usage limit on their $19 plan. I only have the $20 plan from OpenAI and the same task, with a lot of the same implementation details as Kimi Code, only took a few minutes and consumed al…
In my experience, Kimi just tends to think a lot, with the main thing that takes up a lot of space is it constantly second-guessing itself. I've watched it do paragraph after paragraph of "Wait, actually..." while it stumbled and used a ton of tokens on one small detail of what it was asked to do. Though I also gave GLM 5.2 a task to port some JS code to Python to test it, and in my experience it doesn't second guess as bad as Kimi does, but it really did there. It kept doing web searches and second guessing tons of tiny little things, using $0.25 of API spend in total to port about ~50 lines of JavaScript. It did produce an error the first run, but on second run it gave me a program that ran.
I gave Claude Code/Fable the same task and it took significantly less time, but also stumbled on the same error as GLM. I didn't have it fix it though. I was mostly interested in timing differences.
I do like open models where I can, but I'm really hoping they get trained to second guess less. Or maybe I just need to prompt them differently. I'm not sure.
Re: The Kimi K3 Moment
#44This was always where this was heading, but we got here much faster than expected. Once western governments declare it to be a "national security" risk for citizens to have access to open-weight frontier models, and once they classify using these models as acts of terrorism, what will that world be like? Will using Kimi K3 come to be like how napster was in the olden days? Everybody knew it was technically illegal, b…
even 8x rtx pro 6000 is only 768GB of VRAM. IDK how anyone is going to run k3
Re: The Kimi K3 Moment
#45Earlier quoted context omitted.
Is Kimi K3 subsidized as hard as the other models out there?
Does it matter? As an end user I really only care about 1) how much I can do in a week, and 2) how long each task takes. Subsidies would affect 1, but not 2. But if some VC wants to subsidize my Claude or Codex or whatever, awesome.
Subsidization could affect both of those. If you have $200B in the bank you can afford to throw massive compute at every single request; if you are less well funded, you might throttle more aggressively.
Additionally that same VC could be (read: is always) spent on developing the harness, and other infrastructure around the model, not just the model itself.
So it's apples-to-oranges when comparing a relatively new model to established competitors (i.e. OpenAI @ $900B funding vs Moonshot/Kimi's $30B FYI) because every new model they release is judged on "performance" which is not strictly speaking derived solely from the model.
It's possible Moonshot could get similar performance over time as the build out the rest of the infrastructure. We have no way of knowing how much of OpenAI/Anthropic's success is due to the model vs intelligent tooling built on top of it.
Re: The Kimi K3 Moment
#46Regardless of whether they achieved parity via distillation, or whether they got here via independently constructing a model from scratch, it was always going to end this way for the frontier American labs. Distillation “attacks” are not attacks. The frontier labs “distilled” all existing human written knowledge into their models, there was always going to be a second class lab that would distill that model into a ch…
> Distillation “attacks” are not attacks.
If "distillation attacks" happen, we have to conclude there is some value add in what model labs do. Regardless of how we feel about using existing human knowledge in the way they currently do, it's simply impractical to infer that everything that happens downstream of LLMs can not be an attack on some IP because of it.
So both things can be true: a) People infringe on Anthropics IP and b) what Anthropic did to build their models is legally questionable (or might be ruled illegal, even though I doubt it).
Re: The Kimi K3 Moment
#47I tried Kimi K3 on a task I've done with every other model I use regularly ( https://swelljoe.com/post/i-let-every-agent-implement-its-ow... ) and found it chewed a lot longer on the problem and ate up almost the entirety of a 5 hour usage limit on their $19 plan. I only have the $20 plan from OpenAI and the same task, with a lot of the same implementation details as Kimi Code, only took a few minutes and consumed al…
> tried Kimi K3 on a task I've done with every other model I use regularly and found it chewed a lot longer on the problem and ate up almost the entirety of a 5 hour usage limit on their $19 plan
ArtificialAnalysis puts Kimi K3 just below DeepSeek v4 & GLM 5.2 in token use per task, which is about 2x to 3x more tokens than Grok 4.5: https://x.com/ArtificialAnlys/status/2077832879187620192 / https://archive.vn/zBbFi 2 other open weights MiMo v2.5 & MiniMax M3 are comparatively thrifty.
> Subscription usage limits are hard to measure as none of the providers tell you directly what it means in terms of tokens or anything else you can easily compare
I always put my coding subscriptions (that allow it) through "AI gateways" (Cloudflare & OpenRouter are free) which help track token use.
In my experience, Kimi & Qwen Cloud have opaque & restrictive limits, their "credits" drain faster. I now make it a point of subscribing (directly [0]) with providers that are transparent like MiniMax, DeepSeek, Xiaomi, & Z.ai.
[0] OpenCode Go, Cline, and AtlasCloud have generous limits for open weights, otherwise.
Re: The Kimi K3 Moment
#48I think the biggest problem with Chinese models is that they seems to overthink for most of the tasks, especially for smaller ones. The OpenAI models have in my experience only gotten better in terms of efficiency.
Yes, this (imo) is a clear result of benchmaxxing. You can get a much better score on most "intelligence" benchmarks by massively over-saturating reasoning. This looks good on those, but for actual daily usage makes the models much less effective: I don't want a model I use for coding to burn a bunch of reasoning (read: time) on trivial tasks.
It's undeniable that some of these models generate a ton of thinking tokens, but it's arguable whether that makes them "much less effective."
For example, Kimi 2.7 has been really effective for me despite having verbose thinking blocks, simply because it runs so fast. Speed-wise, it feels about like Sonnet, possibly faster.
Re: The Kimi K3 Moment
#49Regardless of whether they achieved parity via distillation, or whether they got here via independently constructing a model from scratch, it was always going to end this way for the frontier American labs. Distillation “attacks” are not attacks. The frontier labs “distilled” all existing human written knowledge into their models, there was always going to be a second class lab that would distill that model into a ch…
> Distillation “attacks” are not attacks. If "distillation attacks" happen, we have to conclude there is some value add in what model labs do. Regardless of how we feel about using existing human knowledge in the way they currently do, it's simply impractical to infer that everything that happens downstream of LLMs can not be an attack on some IP because of it. So both things can be true: a) People infringe on Anthro…
The value is simply that it is easier. The same way it is easier to ask someone who has experience for advice than reading hundreds of textbooks.
Re: The Kimi K3 Moment
#50Earlier quoted context omitted.
https://deepswe.datacurve.ai/ or https://artificialanalysis.ai/ pareto frontier graph.
Thanks! What is the parento frontier?
If you have multiple metrics to evaluate goodness of a design, one would normally need to decide which metrics they care the most about in order to find the "best" design.
The Pareto frontier tells you which designs are the best in at least one of your metrics (non-dominated by another design). For example if you're selecting a car and you care about both speed and mpg, a Formula 1 car and a Prius might lie on the Pareto frontier, but a Model T Ford would not.