Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Kimi K3 Moment

stephen.bochinski.dev

Post reply on HN
Print view View original

431–440 of 644 posts

Re: The Kimi K3 Moment

#431

Post by qlte »

mukmuk wrote: post #375
tristanj wrote: post #348

Earlier quoted context omitted.

API distillation doesn't have to explain all of K3's capabilities for it to have happened. Kimi K3 reproducibly identifies itself as Claude: https://x.com/denisewu/status/2077984660211269870 This behavior is exactly what you'd expect from a model distilled from Claude. There's a detailed analysis of K3's ambiguous identity here: https://github.com/rgreenblatt/which_claude_is_k3/blob/main/... This analysis observed K3…

fwiw, Gemini 3.5 has identified itself to me as an OpenAI product on multiple occasions.

Early Grok would also identify as ChatGPT. This has happened with new model releases for years now.

Re: The Kimi K3 Moment

#432

Post by irthomasthomas »

tristanj wrote: post #413
StrauXX wrote: post #404

Earlier quoted context omitted.

Kimi calling itself claude means nothing. During pre-training, when the model learns to "simulate" the internet text, it will naturally be fed with a bunch of data about Claude and ChatGPT. With the amount of LLM outputs on the internet today, it is not surprising at all that a model would naturally call itself Claude or ChatGPT. You can mitigate that in post-training (or actually in pre-training as well) by training…

Sure, but then Qwen should leak that too, and it doesn't. K3 calls itself Claude 7 out of 48 times, Qwen does it 0 out of 48, and the only other model to identify itself as Claude is DeepSeek. and DeepSeek is alleged to also distill from Claude data anyway. So this isn't something every model absorbed from the same web text. And you skipped over the strongest datapoint that K3 is distilled: K3 reproduces Claude's pub…

Did you know claude models identify as qwen or deepseek when asked in chinese?

Re: The Kimi K3 Moment

#433

Post by manmal »

mikojan wrote: post #411
techjamie wrote: post #43

Earlier quoted context omitted.

In my experience, Kimi just tends to think a lot, with the main thing that takes up a lot of space is it constantly second-guessing itself. I've watched it do paragraph after paragraph of "Wait, actually..." while it stumbled and used a ton of tokens on one small detail of what it was asked to do. Though I also gave GLM 5.2 a task to port some JS code to Python to test it, and in my experience it doesn't second guess…

Interesting. Could you not tell it to not second-guess itself (on minor issues)?

Fighting against the weights is often a lost cause. For any model.

Re: The Kimi K3 Moment

#434

Post by manmal »

ffsm8 wrote: post #160
SwellJoe wrote: post #8

I tried Kimi K3 on a task I've done with every other model I use regularly ( https://swelljoe.com/post/i-let-every-agent-implement-its-ow... ) and found it chewed a lot longer on the problem and ate up almost the entirety of a 5 hour usage limit on their $19 plan. I only have the $20 plan from OpenAI and the same task, with a lot of the same implementation details as Kimi Code, only took a few minutes and consumed al…

Earlier today I made Claude code implement a feature with fable. It worked roughly 60 minutes and used around 30% of my 100€ subs 5h sessions. Then I typed /code-review in a second terminal/clean session after the analysis was done (no code changes) the usage was 99%. I then asked it to write that into a review.md so I could restart from that the next day. Sadly the last % wasn't enough for that. Ymmv, these models b…

Read the session file of the reviewer tomorrow. Sonnet should suffice.

Re: The Kimi K3 Moment

#435

Post by tristanj »

remexre wrote: post #430
tristanj wrote: post #395

Earlier quoted context omitted.

> 'most extensive industrial espionage campaign, probably ever' is absolute nonsense You are completely underestimating the scale of what is happening here. Chinese AI labs are actively facilitating an industrial-scale network of tens of thousands of bot accounts, that resell Claude tokens at 97% below official API prices. They buy subsidized Max 5x plans (sometimes with stolen credit cards), then split the subscript…

if Ford bought hundreds of millions of dollars worth of Hyundais, put extra instrumentation in them, and resold them at a discount to customers who agreed to the instrumentation in exchange for the discount, is Ford doing industrial espionage?

You skipped the part where Ford buys the cars at 90% off and sells them at 80% off, at a profit. Then gets paid by competitors for the driving data.

At the same time, Volvo is running the exact same hustle, except they buy the cars with stolen credit cards, so they get the cars for free.

Re: The Kimi K3 Moment

#436

Post by overfeed »

ACCount37 wrote: post #416
overfeed wrote: post #297

Earlier quoted context omitted.

While it sounds like a lot, do you suppose 3.4 million sessions come even close to being sufficient to train a frontier model? Assuming each session was 10,000 words each, that's 34 billion words; lets call it 50 billion tokens (0.05 trillion) unfairly pilfered from Claude. That left Moonshot needing to scrounge for the other 14.950 trillion training tokens required for a baseline frontier model.

What do you think those tokens are used for? Distillation attacks aren't about replacing the entire pretraining dataset with questionably sourced synthetics. It's all about post-training. Train your own base model - but tune it off Claude output to make it perform more in line with Claude. Yoink the products of Anthropic's expensive SFT, RLHF and RLVR work for yourself by training on the outcomes. The post-training d…

How does yoinking outputs from from prior generation Claude model and post raining on them result in a model competitive with the latest generation? That doesn't add up - nevermind Anthropic hasbeen summarizing thinking tokens since January to counter distillation.

Re: The Kimi K3 Moment

#437

Post by tristanj »

Certhas wrote: post #410
tristanj wrote: post #395

Earlier quoted context omitted.

> 'most extensive industrial espionage campaign, probably ever' is absolute nonsense You are completely underestimating the scale of what is happening here. Chinese AI labs are actively facilitating an industrial-scale network of tens of thousands of bot accounts, that resell Claude tokens at 97% below official API prices. They buy subsidized Max 5x plans (sometimes with stolen credit cards), then split the subscript…

You haven't explained how this is illegal or any more immoral than scraping the web for training data. As you said yourself: They are buying the product. Then they are using it for their own purposes. That's more than Anthropic/OpenAI did for the open internet. That's more than Meta did when they obtained torrents of books in the early days, and then claimed that even though the data was obtained illegally they can s…

> They paid for it!

They didn't though. The resellers are not buying via the official API, they're buying Max subscriptions (where tokens are priced ~10x below API cost), then splitting the subscription across dozens of clients and reselling the output as the regular API. Anthropic prices its subscription plans barely at cost, to bring in customers onto their enterprise plans where they can charge expensive API rates. Reselling these subsidized plans for price arbitrage is a TOS violation. It's not a legitimate purchase. Plus, a non-trivial amount of this volume is funded by stolen credit cards, so this "revenue" gets chargeback anyway.

The resellers then log all the the model output, then sell it to Chinese labs as training data.

> is it any more immoral than scraping the web for training data.

I think you'd acknowledge there's a difference between "We indexed public web pages" and "We deployed tens of thousands of fraudulent accounts to resell your subsidized plan for cheap, stealing your own customers, while collecting the data to build our own competing product" are very different actions. One can believe the first was wrong while acknowledging the second is far worse.

Re: The Kimi K3 Moment

#438

Post by irthomasthomas »

tristanj wrote: post #373
remexre wrote: post #369

Earlier quoted context omitted.

my first prompt to any Kimi model was K3 via Pi, some version of "hi kimi!!" and the response was telling me "I'm actually Claude." this is not hard to repro, just use a system prompt that doesn't mention the model name. that said, if they bootstrapped with opus 4.6 convo sft data they had sitting around... so what?

The main story is what isn't being talked about. Chinese labs exfiltrated trillions of tokens of high-quality output from Anthropic and OpenAI, through proxies and heavily discounted token resellers, which they distilled and used for training data for their own models. Instead of spending 12-18 months building their own robust harnesses and painstakingly creating quality training data (which is what Anthropic and Ope…

Ask claude its name in Chinese and it says Qwen or Deepseek. Anthropic distilled Chinese tokens rather than create their own Chinese language training data.

Re: The Kimi K3 Moment

#439

Post by Ey7NFZ3P0nzAe »

rzerowan wrote: post #28
montroser wrote: post #13

This was always where this was heading, but we got here much faster than expected. Once western governments declare it to be a "national security" risk for citizens to have access to open-weight frontier models, and once they classify using these models as acts of terrorism, what will that world be like? Will using Kimi K3 come to be like how napster was in the olden days? Everybody knew it was technically illegal, b…

Basically a new iron curtain didving the world into digiatl blocks.The era of open internet/science is on its last legs with the potential forr bifurcation into incompatible ecosystems high , the onger the exchange is disrupted. As recently as this month the USgov has donce a Wolf Amendment style declaration for the Scientific collaboration NSF while shifting its purview under the military. To add to that its trying…

> The Wolf Amendment is a law passed by the United States Congress in 2011, named after Representative Frank Wolf, that prohibits the National Aeronautics and Space Administration (NASA) from using government funds to engage in direct, bilateral cooperation with the Chinese government and China-affiliated organizations from its activities without explicit authorization from the Federal Bureau of Investigation and Congress

https://en.wikipedia.org/wiki/Wolf_Amendment

Re: The Kimi K3 Moment

#440

Post by freely0085 »

Danox wrote: post #116
rzerowan wrote: post #28

Earlier quoted context omitted.

Basically a new iron curtain didving the world into digiatl blocks.The era of open internet/science is on its last legs with the potential forr bifurcation into incompatible ecosystems high , the onger the exchange is disrupted. As recently as this month the USgov has donce a Wolf Amendment style declaration for the Scientific collaboration NSF while shifting its purview under the military. To add to that its trying…

At this point, the United States will lose that battle most Countries in the world are going end up using electronics from Asia, that ship has sailed Japan, China, Korea, Singapore, Taiwan, Vietnam, dominate that area, China already dominates EVs, Drones and many other electronic devices, and with the way Donald Trump has picked fights, Europe, Canada, Australia, New Zealand, Mexico and many others are looking for ot…

Singapore? Singapore doesn't produce shit.
Post reply on HN