Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Kimi K3 Moment

stephen.bochinski.dev

Post reply on HN
Print view View original

391–400 of 644 posts

Re: The Kimi K3 Moment

#391

Post by 6510 »

disentanglement wrote: post #383
tristanj wrote: post #373

Earlier quoted context omitted.

The main story is what isn't being talked about. Chinese labs exfiltrated trillions of tokens of high-quality output from Anthropic and OpenAI, through proxies and heavily discounted token resellers, which they distilled and used for training data for their own models. Instead of spending 12-18 months building their own robust harnesses and painstakingly creating quality training data (which is what Anthropic and Ope…

I could perhaps get myself to care just the tiniest bit if the information that was supposedly stolen wasn't generated by "stealing" from everybody else. Either it is fair use to train AI models on whatever information you can get your hands on for everyone or for no one.

Stack overflow is pretending to be Claude now. I wonder if one can get it to say your question had already been asked.

Re: The Kimi K3 Moment

#392

Post by JSR_FDED »

nickysielicki wrote: post #33

Regardless of whether they achieved parity via distillation, or whether they got here via independently constructing a model from scratch, it was always going to end this way for the frontier American labs. Distillation “attacks” are not attacks. The frontier labs “distilled” all existing human written knowledge into their models, there was always going to be a second class lab that would distill that model into a ch…

The court decided that LLMs are a transformative fair use of the data they trained on, and therefore aren’t copyright infringement.

Maybe Kimi is a derivative work as well

Re: The Kimi K3 Moment

#393

Post by Grimblewald »

tristanj wrote: post #348
anon373839 wrote: post #223

Earlier quoted context omitted.

I strongly agree with the premise that distillation is not an “attack”. But that said: K3 is not a distilled version of Fable or Sol. Fable has been barely available and Sol was just released! Moreover, K3 is superior to both models in some domains, according to user scoring on the Arena. API distillation can’t give you these results anyway. All it is useful for is bootstrapping RL in new domains to get past the “col…

API distillation doesn't have to explain all of K3's capabilities for it to have happened. Kimi K3 reproducibly identifies itself as Claude: https://x.com/denisewu/status/2077984660211269870 This behavior is exactly what you'd expect from a model distilled from Claude. There's a detailed analysis of K3's ambiguous identity here: https://github.com/rgreenblatt/which_claude_is_k3/blob/main/... This analysis observed K3…

and claude will call itself chatgpt etc.

nothing new, all ai labs are immoral and not bound by any reasonable oversight or ethical constraints. All outlaws in their own rights on that front. Absolutely none of them have true rights on the matter of being distilled from given historic and continued behaviour. I'm not sure why this is a talking point at all? We know AI companies steal, the least interesting behaviour among this is them stealing from one another.

For me, a far more interesting and important point of conversation on this matter is anthropic buying rare or evwn unique books, processing them for training data, and then destroying the books for others cannot use it as well.

Permanemt destruction of priceless primary source materials is so many leagues beyond copying a copy that I cannot fathom it even registering as a discussion point.

Re: The Kimi K3 Moment

#394

Post by amunozo »

darkoob12 wrote: post #354
rayiner wrote: post #336

Earlier quoted context omitted.

The desire to accuse China of just copying is like 20 years out of date. It’s been wrong since some people on HN were in diapers. People are going to be gobsmacked when, in our lifetime, China becomes a world power comparable to the U.S. Probably still poorer per capita, but at Spain/Italy levels, not third world country levels. And they’ll be shocked at the implications of that on the world economy, migration patter…

There are huge evidence of copying. Some day China can pioneer in science or technology but the current claim about Chinese companies leading AI development is ridiculous given the evidence of distillation and the fact that like 95 percent of science that lead to the current state of AI happened in either North America or Europe. To be honest if you want to list academic papers that lead to the current AI models the…

What about EVs, solar, or batteries? They are leading these fields for some time.

Re: The Kimi K3 Moment

#395

Post by tristanj »

vanviegen wrote: post #382
tristanj wrote: post #373

Earlier quoted context omitted.

The main story is what isn't being talked about. Chinese labs exfiltrated trillions of tokens of high-quality output from Anthropic and OpenAI, through proxies and heavily discounted token resellers, which they distilled and used for training data for their own models. Instead of spending 12-18 months building their own robust harnesses and painstakingly creating quality training data (which is what Anthropic and Ope…

Shrug. Hard to feel sorry for companies that created their empires by ignoring copyright themselves. Also, 'most extensive industrial espionage campaign, probably ever' is absolute nonsense. They did not need to infiltrate the companies for this nor are you accusing them of stealing any trade secrets. This is only about whether they looked at their competitors' products from the outside (in the form of conversation t…

> 'most extensive industrial espionage campaign, probably ever' is absolute nonsense

You are completely underestimating the scale of what is happening here.

Chinese AI labs are actively facilitating an industrial-scale network of tens of thousands of bot accounts, that resell Claude tokens at 97% below official API prices. They buy subsidized Max 5x plans (sometimes with stolen credit cards), then split the subscription across dozens of clients and reselling the output. They are running a massive data-harvesting operation. Chinese labs and token resellers subsidize the cost of the tokens in exchange for the API metadata (detailed reasoning traces, model outputs, and tool calls) to use as high-quality training data for their own models.

They are buying Anthropic's own product, just to resell it below cost, just so they can capture the training data. Reportedly, they are paying as much as ~$0.01 per tool call.

https://x.com/yan5xu/status/2029743983522631698

I explained what is happening in this thread: https://news.ycombinator.com/item?id=48664814

Re: The Kimi K3 Moment

#396

Post by JSR_FDED »

darkoob12 wrote: post #354
rayiner wrote: post #336

Earlier quoted context omitted.

The desire to accuse China of just copying is like 20 years out of date. It’s been wrong since some people on HN were in diapers. People are going to be gobsmacked when, in our lifetime, China becomes a world power comparable to the U.S. Probably still poorer per capita, but at Spain/Italy levels, not third world country levels. And they’ll be shocked at the implications of that on the world economy, migration patter…

There are huge evidence of copying. Some day China can pioneer in science or technology but the current claim about Chinese companies leading AI development is ridiculous given the evidence of distillation and the fact that like 95 percent of science that lead to the current state of AI happened in either North America or Europe. To be honest if you want to list academic papers that lead to the current AI models the…

From the Hoover Institution’s analysis of the team behind DeepSeek:

“We find striking evidence that China has developed a robust pipeline of homegrown talent. Nearly all of the researchers behind DeepSeek’s five papers were educated or trained in China. More than half of them never left China for schooling or work, demonstrating the country’s growing capacity to develop world-class AI talent through an entirely domestic pipeline. And while nearly a quarter of DeepSeek researchers gained some experience at US institutions during their careers, most returned to China, creating a one-way knowledge transfer that benefits China’s AI ecosystem.”

That was from a year ago.

Consider that on top of this the country was starved of access to Nvidia chips - and therefore accelerated its development of Ascend chips, and it’s clear they are undeniably leaders in AI research and development. Not the only ones, but the achievements are crystal clear.

Re: The Kimi K3 Moment

#398

Post by monkey_monkey »

tristanj wrote: post #373
remexre wrote: post #369

Earlier quoted context omitted.

my first prompt to any Kimi model was K3 via Pi, some version of "hi kimi!!" and the response was telling me "I'm actually Claude." this is not hard to repro, just use a system prompt that doesn't mention the model name. that said, if they bootstrapped with opus 4.6 convo sft data they had sitting around... so what?

The main story is what isn't being talked about. Chinese labs exfiltrated trillions of tokens of high-quality output from Anthropic and OpenAI, through proxies and heavily discounted token resellers, which they distilled and used for training data for their own models. Instead of spending 12-18 months building their own robust harnesses and painstakingly creating quality training data (which is what Anthropic and Ope…

Why should anyone care? I couldn't give a single fuck, in fact if what you assert is true (definitely not proven), I applaud Moonshot - seems like a very smart way to operate.

Re: The Kimi K3 Moment

#399

Post by fnordpiglet »

mappu wrote: post #211
loopmonster wrote: post #20

Earlier quoted context omitted.

This line made me think by 'normal coding work' the author means doing something they don't understand well enough to be able to distinguish the models' output.

Steve Yegge calls this is the "discernment horizon" - https://steve-yegge.medium.com/the-flat-curve-society-36c8b0...

I think a better framing is the marginal utility of the models capability growth. At a certain point frontier models will only be needed for frontier problems. The demand for that capability will decrease with time. The hand wringing about not understanding is to my mind anthropomorphic - AI of today lack agency and awareness. Even the constructed stuff Anthropic puts out there in the model docs involve contrived scenarios to elicit “scary” behaviors. It’s unclear that as models become more sophisticated whether they’re better at instruction following or not but it certainly feels that way - even if it’s through better alignment or just an artifact of scaling. However I think the malign actors of humans using powerful models for bad stuff isn’t unreasonable to be concerned about.

The marginal utility problem is a real one for AI companies. I think the current generations are already saturating marginal utility for 95% of the population. Almost everyone I know outside of my career has no use for a more powerful model. This is a serious problem for the economics of AI and semiconductor investment. This is a bigger problem than Chinese models. It leads to a demand curve problem - that supply outstrips demand.

Re: The Kimi K3 Moment

#400

Post by JSR_FDED »

areoform wrote: post #292
boondongle wrote: post #64

Earlier quoted context omitted.

Almost all markets depend on some form of regulation whether its as simple as "leave everyone alone but no stealing" or "every participant has to source every object through mountains of red tape." Thus far the US has not really chosen to go the Chinese rare-earth method yet. The problem with distillation attacks is the end result is everyone who is not doing them is going to deal with some kind of regulation whether…

> The problem with distillation attacks I think it's worth stepping back here and pointing out the obvious. Y'all waging war on math. And I'm sorry, but that's the computing equivalent of legislating gravity. Apologies for repeating myself here, but what you call "distillation" is function approximation. I feel for the teams at Anthropic and Open AI, but unlike startups from prior eras; Anthropic and OpenAI have deci…

I like your point that there is so much content being created by LLMs that at some point there’s enough to perform something like distillation without even needing to interact with the LLMs directly.
Post reply on HN