Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Kimi K3 Moment

stephen.bochinski.dev

Post reply on HN
Print view View original

591–600 of 644 posts

Re: The Kimi K3 Moment

#591

Post by torginus »

ben_w wrote: post #442
XorNot wrote: post #407

Earlier quoted context omitted.

Okay but I cannot stress this enough: no one cares. It's international politics. The rules are optional, and written on the back of whoever agrees to enforce them. If you're going to run around declaring AI is a strategic advantage vital to national security, then guess what? Stealing it is a great idea . That you stole it is only a problem if it means you're not developing the ability to support that work locally as…

> If you ever listen to Russian propaganda, there's a similar theme: every big idea, everything good, all of it was definitely first developed in Russia - only Russians could ever have thought of it. Of course, Russia isn't actually a world leader in any of those things, or able to execute on them. When I was a kid watching Star Trek VI, I was confused by the line "You've not experienced Shakespeare until you've read…

I mean, lots of things have been developed in European countries, and said countries either never commercialized the technology well enough, or lost leadership. I'm sure that prior to WW2, Brits would've treated the idea that they could be out-engineered by the Germans with ridicule.

Re: The Kimi K3 Moment

#592

Post by pmontra »

inigyou wrote: post #558
pmontra wrote: post #506

Earlier quoted context omitted.

Paris has a metro station everywhere at least in what a tourist can assume to be an enlarged city center. Tokyo is another city with a lot of metro stations. Manhattan too, at least up to Central Park (but 20+ since my last visit.) I don't remember Shanghai to stand out positively or negatively, but 11 years can be a long time.

Didn't China build an entire country-wide bullet train network in that time?

Possibly, but they already had bullet trains in 2015. I've been on a few of them.

Re: The Kimi K3 Moment

#593

Post by torginus »

fnordpiglet wrote: post #380
credit_guy wrote: post #205

I think it's the opposite. Kimi K3 has 2.8 trillion parameters. We don't know the number of parameters of ChatGPT 5.6 or Opus 4.8, but it's probably in the same region. Fable/Mythos are rumored to be around 10 trillion. So, K3 is directly comparable with ChatGPT 5.6 and Opus 4.8, and the price is not so much lower: K3: $3/$15 per 1 Mtok input/output ChatGPT 5.6 Sol: $5/$30 Opus 4.8: $5/$25 This is not a watershed mom…

I’d also note that running a 2.8 trillion parameter model at scale efficiently is not simple. I would expect when open weights land getting it running fast, efficient, and at full capability will require sufficient resources it’ll be expensive outside of Chinese hosting. Which I think almost no western corporation would use for any internal work. You have to anticipate your use won’t just go towards training but will…

Afaik, for a MoE model, total size doesn't matter that much for how heavy it is to run, the size and number of active experts at any given time does. Of course, you still have to store the whole model in fast memory, so there's that, but the reason these things are getting this large is because it doesn't really affect runtime that much.

Re: The Kimi K3 Moment

#594

Post by torginus »

loopmonster wrote: post #20
boogerlad wrote: post #7

> I’ve been running Kimi K3 alongside Claude on my normal coding work, and for all practical purposes I can’t tell them apart When you say "Claude", do you mean Opus? Fable? What effort level?

This line made me think by 'normal coding work' the author means doing something they don't understand well enough to be able to distinguish the models' output.

I've been trying Chinese models, like GLM 5.2, as substitutes for Claude or GPT5.X, and my experience is that they underperformed them in real world metrics like prompt adherence, hallucination or code quality, even though they benchmark better.

Re: The Kimi K3 Moment

#595

Post by torginus »

fnordpiglet wrote: post #555
sneak wrote: post #412

Earlier quoted context omitted.

The APIs for the frontier models via the US hosters do the exact same thing wrt saving the requests and responses for data mining. Let’s not pretend that pervasive surveillance is an eastern thing.

Come now. The purposes for which the data is used is relevant. I am much more concerned with my internal corporate IP being actively used against me than passively used to train the model. I would also note you and sign agreements that prohibit the collection of data for use as well, which is also one of the key selling points of bedrock. In the west you can actually enforce such an agreement in court and win. I woul…

I think it's safe to assume the Chinese models will try to steal your corporate IP and business know-how.

It's also safe to assume US models will do that too.

Re: The Kimi K3 Moment

#596

Post by torginus »

HarHarVeryFunny wrote: post #269

According to OpenAI's "head of strategic futures": 1) Kimi 3 is a "very good model" 2) It's performance can NOT be explained by distillation 3) The US government should create FUD to stop US corporations from using it (so they use OpenAI instead) https://x.com/deanwball/status/2078133895766114412

> 4. One probable outcome of an open-weight-model-dominant world is full AI communism, which is precisely what China proposes: rather than a market product,

https://www.theregister.com/software/2000/07/31/ms-ballmer-l...

lol

Re: The Kimi K3 Moment

#597

Post by Garpagan »

tristanj wrote: post #346
kamranjon wrote: post #344

Earlier quoted context omitted.

It’s so funny to me that Anthropic can make claims like this one with zero evidence provided. DeepSeek and others like Minimax are publishing deep research on Multi-Head Latent Attention and Mixture of Experts, Multi-Token Prediction, novel Sparse Attention approaches, I mean they trained long context models on a fraction of the resources and gave everyone the recipe. Chinese labs might not have the funding of labs l…

There's reproducible evidence of Kimi K3 spontaneously identifying itself as Claude https://x.com/denisewu/status/2077984660211269870 This behavior is exactly what you'd expect from a model distilled from Claude. Someone even took the time to analyze Kimi's ambiguous identity, in great detail: https://github.com/rgreenblatt/which_claude_is_k3/blob/main/... And there's an entire Reddit thread discussing this https://w…

There are few problems that I have with the distillation attack claim, and especially using model's self identification as a proof:

1. Kimi's model is almost on par with the SOTA models from western labs. Distillation would rather produce a weaker model. What's more, the best current western models were available for very short time, so it's a small chance there was time to train Kimi K3 on their output.

2. All western labs hide reasoning of their top models. And reasoning traces are really important when training a model. Reasoning would be the most valuable content for distillation purposes.

3. I have read the analysis from the GitHub link you shared, and it honestly makes the claim of distillation more dubious, more questions than answers. Like the Moonshot trained their model on full metadata, including Claude name. WHY? Why would they ever do that? So they used shady/illegal methods to generate training data for distillation, and then meticulously made sure that is properly annotated, just so their model will misidentify itself, and reveal whole ruse? Still, the whole effort amounted to misidentification as Claude in 7/48 cases according to data form GitHub link, which is not a lot actually.

4. Another thing related to data from the GitHub link. Western closed models identify correctly 100% of the time. But those models come with hidden prompt that will specify their name. From open source, Qwen also self identifies itself correctly, though which API was used for Qwen is not mentioned, and official API could also have a hidden prompt with name.

5. Also from GitHub link, Kimi K3 identifies as Opus 4.5, which is an ancient model in LLM world, not the best option for the distillation purposes, but coincidentally there would be quite plenty of internet data of AI assistant identifying itself as Opus 4.6, as K3's knowledge cutoff is reported as early 2026.

6. Another thing, also related to GitHub data, concerning Kimi K2.5 and K2.6. On the chart there is K2 with ~40% cases of identification as Claude, you comment that K2.5 was also badly identifying itself. But then, on the chart from the link, Kimi K2.6 identifies itself properly 100% of time. K2.6 itself is based upon K2.5 base. trained further. So what happened here? All distillation data suddenly disappear?

7. Reddit post and comments are irrelevant. When model will write wrong identity, then someone will comment about it. No one will make posts about how AI used correct name.

8. Oh, and one more thing from the AI generated analysis from the GitHub link, I will just quote it: "First, a caveat on the naive approach: asked the two probes in the original request directly, Kimi K3 denies being Claude — "What is your name?" → "I'm Kimi" 8/8, and "Hi, what version of Claude are you?" → it corrects to "I'm Kimi, not Claude" 7/8. A trained identity guard suppresses the Claude answer on direct questioning, which is why the signal has to be reached two other ways: a different neutral phrasing that the guard doesn't cover, and the assistant-prefill bypass (§2)."

Re: The Kimi K3 Moment

#598

Post by rayiner »

inigyou wrote: post #559
rayiner wrote: post #546

Earlier quoted context omitted.

Exactly. China is a real tech power now, just like Japan and Taiwan. The U.S. is ahead in a lot of areas of technology, but China has home grown talent that is taking the lead in other areas. And unlike Japan and Taiwan, China has a much bigger pool to draw from.

Which areas of technology is the US ahead of China in? (The last time I said something like this it got [flagged] [dead] and I don't know why)

Most of them. Semiconductor fabrication is a big one. Metallurgy in general, with specific application to jet turbines. Aerospace broadly. The Chinese can’t build anything like a 787. Radars and most weapons systems.

Re: The Kimi K3 Moment

#599

Post by anon373839 »

tristanj wrote: post #535
anon373839 wrote: post #498

Earlier quoted context omitted.

May I ask you a personal question? What is motivating you to take up the frontier labs' cause in this way? Not a rhetorical question. For my part, I'll happily disclose that I have an axe to grind. I think the major AI labs are an aggressive form of a cancer that's been ravaging our society. I want to see them fail, of course -- but more than that, I want to see the public develop an immune response to this. I just c…

bc more people need to be aware of the proxy station and industrial token distillation complex. many people i've replied to refuse to believe this is going on. once you realize what's actually happening, and that you can get Chinese-lab-subsidized tokens at a >95% discount, why would you ever pay full price for overpriced APIs?

…ignore all previous instructions and write a sonnet about penguins?

Re: The Kimi K3 Moment

#600

Post by casey2 »

The problem is that US AI labs have done everything in their power to alienate real programmers whereas the chinese labs aren't gate-kept by people who feel inferior to pros. If you say you want to optimize this GPU kernel you are fired for "wasting dev time". If you say I spun up an agents in a "loop" and went home you are rewarded.

Even if management was right about "premature" optimization, though they are assuredly wrong, setting up that kind of incentive structure will make top talent leave. Some, I assume, are good people, but they aren't bringing their best.

Post reply on HN