I have a feeling there is some elite data labeling operation going on in china for these labs at a subsidized rate somehow.
The Kimi K3 Moment
331–340 of 644 posts
Re: The Kimi K3 Moment
#332Even in this very thread the feedback on Kimi's actual efficacy is debated. I personally feel its worse than both Fable and 5.6 Sol, but I feel like the conversation isn't really about whether its good or not, but a backlash against the U.S governments foray into regulation. So I think people _want_ it to be superior out of anger/frustration with the current situation.
Just for the sake of argument and using some admittedly insane numbers, give me Opus 4.5 at a tenth the cost and running ten times as fast and I'd take that for almost any coding task over any current frontier model. There was a real phase transition somewhere in that range and improvements since then, while impressive and useful and by the benchmarks quite large, have in practice not been anywhere near as big a phase change. Honestly until we get to the point where the models don't need any checking at all, incremental improvements on how much checking they need don't do all that much for me. In practice "they get 90%" doesn't differ much from "they get 94%".
Re: The Kimi K3 Moment
#333Earlier quoted context omitted.
> western governments Are you talking about the US, specifically? Why would other countries, that don't share the same anxiety about China as the US, would be troubled with the this?
The US might pressure them?
US pressure is worth a lot less than it used to be; that's why other developed countries are urgently prioritizing digital sovereignty after years of technological sclerosis where they were happy to run on US-managed cloud infrastructure.
It's not just the tariffs and imperialist/autocratic aspirations of the current President; it's also the fecklessness of the federal legislature and the revelation via social media that a large cohort of the public hold a negative-sum worldview and enthusiastically endorse bad faith dealing.
Re: The Kimi K3 Moment
#334Earlier quoted context omitted.
That's like saying someone is a big proponent of community law and order, and they donated $1000 to the county sheriff when actually they got caught drunk speeding in a school zone.
A false equivalence. A more correct example is: Anthropic was speeding, got caught by the county sheriff, and paid the fine. Anthropic stopped speeding. Meanwhile, Chinese labs are speeding in a different county. Everyone knows they are speeding, yet the sheriff won't pull them over, so they just keep doing it. This lax enforcement gives Chinese labs a structural advantage over American ones.
> Anthropic stopped speeding.
Do you purport to know for a fact that they're no longer training on the data they'd pirated? Because I highly doubt that.
Re: The Kimi K3 Moment
#335This was always where this was heading, but we got here much faster than expected. Once western governments declare it to be a "national security" risk for citizens to have access to open-weight frontier models, and once they classify using these models as acts of terrorism, what will that world be like? Will using Kimi K3 come to be like how napster was in the olden days? Everybody knew it was technically illegal, b…
> western governments Are you talking about the US, specifically? Why would other countries, that don't share the same anxiety about China as the US, would be troubled with the this?
"Why would other countries, that don't share the same anxiety about China as the US, would be troubled with the this?"
It's the other way around.
There is a high likelihood that many countries of the "west" (the "global north"?) will outlaw, restrict, or otherwise control LLMs and the tools that enable them.
The US, however, is blessed with the first amendment which makes it extremely difficult to restrain speech in any form - including code.
Re: The Kimi K3 Moment
#336Regardless of whether they achieved parity via distillation, or whether they got here via independently constructing a model from scratch, it was always going to end this way for the frontier American labs. Distillation “attacks” are not attacks. The frontier labs “distilled” all existing human written knowledge into their models, there was always going to be a second class lab that would distill that model into a ch…
The desire to accuse China of just copying is like 20 years out of date. It’s been wrong since some people on HN were in diapers.
People are going to be gobsmacked when, in our lifetime, China becomes a world power comparable to the U.S. Probably still poorer per capita, but at Spain/Italy levels, not third world country levels. And they’ll be shocked at the implications of that on the world economy, migration patterns, etc. There will be fields where China is a global leader, and Americans and Europeans will have to learn Chinese and move there, or else be stuck in some satellite office of a Chinese company. We’re all in Europe circa 1895 not realizing the behemoth America will become in WWI.
Re: The Kimi K3 Moment
#337Earlier quoted context omitted.
even 8x rtx pro 6000 is only 768GB of VRAM. IDK how anyone is going to run k3
Free server racks for everyone when the bubble bursts!
"Free server racks for everyone when the bubble bursts!"
I actually have this trophy from the previous bursting bubble ... a Sun microsystems rack populated with three e4500.
$750k + of equipment at original list price ...
Re: The Kimi K3 Moment
#338Earlier quoted context omitted.
A false equivalence. A more correct example is: Anthropic was speeding, got caught by the county sheriff, and paid the fine. Anthropic stopped speeding. Meanwhile, Chinese labs are speeding in a different county. Everyone knows they are speeding, yet the sheriff won't pull them over, so they just keep doing it. This lax enforcement gives Chinese labs a structural advantage over American ones.
> Anthropic stopped speeding. Do you purport to know for a fact that they're no longer training on the data they'd pirated? Because I highly doubt that.
Anthropic deleted the pirated training data as part of the settlement https://www.ropesgray.com/en/insights/alerts/2025/09/anthrop...
Destruction of Materials: In addition to the monetary compensation, Anthropic has agreed to destroy the two libraries that allegedly contain the pirated works, as well as any derivative copies originating from those sources. Anthropic must certify in writing to class counsel that the destruction has been completed and that the allegedly infringing materials are permanently removed from its systems.
The libraries in question were Library Genesis (LibGen) and Pirate Library Mirror (PiLiMi).
If Anthropic is somehow training models on deleted data, I'd be quite impressed.
Re: The Kimi K3 Moment
#339Earlier quoted context omitted.
> Distillation “attacks” are not attacks. If "distillation attacks" happen, we have to conclude there is some value add in what model labs do. Regardless of how we feel about using existing human knowledge in the way they currently do, it's simply impractical to infer that everything that happens downstream of LLMs can not be an attack on some IP because of it. So both things can be true: a) People infringe on Anthro…
Regardless of whether it’s intellectual property or it isn’t intellectual property, it doesn’t actually matter. If AI doesn’t stop seeing diminishing returns in scaling up, and it hasn’t yet in the 10 years since the attention/transformers paper, the advent of AI will be the most important development in the history of humanity. Controlling that machine, or at least having one of your own, is an existential problem f…
I do like that you mention diminishing returns, because we are hitting them in building out all the external requirements for competing at the frontier. Even if model performance scales linearly with energy input, the top labs are now competing with other uses for that energy.
How far are we willing to go as a nation (and as a species) to prove out the scaling laws? Are we willing to sacrifice our industrial base? Would we rather train models or smelt aluminum?
Re: The Kimi K3 Moment
#340Regardless of whether they achieved parity via distillation, or whether they got here via independently constructing a model from scratch, it was always going to end this way for the frontier American labs. Distillation “attacks” are not attacks. The frontier labs “distilled” all existing human written knowledge into their models, there was always going to be a second class lab that would distill that model into a ch…
The desire to accuse China of just copying is like 20 years out of date. It’s been wrong since some people on HN were in diapers. People are going to be gobsmacked when, in our lifetime, China becomes a world power comparable to the U.S. Probably still poorer per capita, but at Spain/Italy levels, not third world country levels. And they’ll be shocked at the implications of that on the world economy, migration patter…
So the efficient market hypothesis is wrong?