Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

bloomberg.com

Post reply on HN
Print view View original

61–70 of 79 posts

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#61

Post by HarHarVeryFunny »

nomel wrote: post #59
corranh wrote: post #21

Earlier quoted context omitted.

I’m still baffled that OpenAI can complain about this with a straight face while these models are literally trained on everything regardless of copyright or permission.

Raw stolen data that is in no way related to AI vs a very very expensive transformative compilation of that raw stolen data that results in usable AI. The frustration is completely understandable to me, since distillation skips much of that "very very expensive" part. And yes, I understand the stolen data was expensive to make, so I understand the owners of it are also frustrated, but that's partly a problem with cur…

Not really, in this context the word "distillation" is really being abused, or at least used in a different sense than when it was originally introduced in the Hinton et al "Distilling the Knowledge in a Neural Network" paper, where it was essentially referring to knowledge compression.

The way Anthropic are using "distillation" is just in a very broad vague sense to claim that some data generated by their model was used to help train another one. They are not talking about something like internal logits, expensive to derive, that would be useful to train a smaller model, but rather about any output from their model, even outputs with redacted reasoning (i.e. incomplete outputs that do NOT reflect the underlying knowledge of the source model).

Given the way Anthropic are using the word, IMO it's better just to think of this as cheap training data, and indeed very similar to the way Anthropic themselves got cheap training data just by taking it (even in cases where that was illegal - copyright). The alternative for Moonshot would be to pay for human generated reasoning data, just as the alternative for Anthropic would have been to pay human developers for coding data etc, not just take it from wherever they could lay their hands on it.

So, I guess Moonshot may have violated Anthropic's TOS, in using Anthropic output to compete against Anthropic, but unlike Anthropic they at least didn't break copyright law since model output is not copyright, and technically they may not have even violated Anthropic's Terms of Service unless they owned the accounts used to access Anthropic's models (perhaps not - they may have used one of the anonymizing Chinese token resellers).

So yeah - pot calls kettle black.

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#62

Post by basiccalendar74 »

ycui7 wrote: post #11

K3 is natively trained to mxfp4, if they cannot get a hold of Blackwell chip, it is meaningless. Hopper does not do native 4-bit floating math. Either they have Blackwell with native 4-bit floating math, or they use have Chinese domestic NPU that support mxfp4 natively. The article’s statement does not make sense.

Weights will be fetched 4-bit from memory, but compute happens in FP8 (W4A8) or BF16 (W4A16). This is still better than fetching 8-bit weights from memory.

Both W4A8 and W4A16 schemes are supported by Hopper GPUs and commonly used to serve mxfp4 Kimi models on Hopper.

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#63

Post by trollbridge »

radialstub wrote: post #17
trollbridge wrote: post #10

Earlier quoted context omitted.

Getting smaller is how the last revolution in computing happened. A VAX 11/780 was good, but an 80386 was a lot better, since the latter could run on 3 AA batteries and the former needed 6,000 watts of 3 phase.

Yes but that's because of the scaling laws for transistors. ML models seem to get better the bigger they are. If you want to compress the world's information, you need to look at all the information in the world.

Qwen 3.6 27b is way ahead of Llama 70b.

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#64

Post by UqWBcuFx6NV4r »

andy_ppp wrote: post #41
HarHarVeryFunny wrote: post #2

Interesting if true - that Moonshot can train a ~3T SOTA model on only 20K NVIDIA GPUs, while others like Musk (who freely admits to distilling OpenAI's models) struggle to build a competitive 1T model (Grok 4.5) with massively more compute (Colossus-1 100-250K GPUs, Colossus-2 500K+ GPUs). I guess at least partly a reflection of all the optimizations in the Kimi 3 architecture. In the recent leaked DeepSeek investor…

Do we believe them? It seems there’s no reason with the resources OpenAI and Anthropic have they wouldn’t be incentivised to build equally efficient pipelines? I think it’s mostly the fine tuning with distillation that gives Chinese labs their big advantage. However I’m pretty sure all the main labs are poisoning the output now when they detect Chinese activity so while K3 looks frontier on tests when you use it it’s…

“The only reason that China is good is because it steals from us” spoken from atop the unambiguously crumbling has-been empire.

I’m not American nor Chinese, and I’m from a much more US-aligned country. But, Christ, you lot really are asking for it.

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#65

Post by andy_ppp »

UqWBcuFx6NV4r wrote: post #64
andy_ppp wrote: post #41

Earlier quoted context omitted.

Do we believe them? It seems there’s no reason with the resources OpenAI and Anthropic have they wouldn’t be incentivised to build equally efficient pipelines? I think it’s mostly the fine tuning with distillation that gives Chinese labs their big advantage. However I’m pretty sure all the main labs are poisoning the output now when they detect Chinese activity so while K3 looks frontier on tests when you use it it’s…

“The only reason that China is good is because it steals from us” spoken from atop the unambiguously crumbling has-been empire. I’m not American nor Chinese, and I’m from a much more US-aligned country. But, Christ, you lot really are asking for it.

I don’t think you can steal someone’s work if it’s based entirely off theft tbh, I see distillation as smart - are you denying it’s a huge part of why Chinese models are competitive or do you believe the narrative yourself that they are doing this with several orders of magnitude less compute and the US labs are profligate and fond of burning money rather than optimising?

The truth is probably Anthropic/OpenAI/Google are pretty efficient but less efficient than the Chinese labs, the Chinese labs probably have more compute than they say to undermine US spending and distillation is quite efficient at bridging the gap in compute.

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#66

Post by hereme888 »

UqWBcuFx6NV4r wrote: post #64
andy_ppp wrote: post #41

Earlier quoted context omitted.

Do we believe them? It seems there’s no reason with the resources OpenAI and Anthropic have they wouldn’t be incentivised to build equally efficient pipelines? I think it’s mostly the fine tuning with distillation that gives Chinese labs their big advantage. However I’m pretty sure all the main labs are poisoning the output now when they detect Chinese activity so while K3 looks frontier on tests when you use it it’s…

“The only reason that China is good is because it steals from us” spoken from atop the unambiguously crumbling has-been empire. I’m not American nor Chinese, and I’m from a much more US-aligned country. But, Christ, you lot really are asking for it.

Phew, if that's not clearly anti-American hate, insult towards Christianity, hatred in general.... Then idk what is.

And btw, yes it's an objective fact from every technical angle that the Chinese only have competitive models because of US tech. Why do you think they try so hard to smuggle NVIDIA GPUs, and now exposed infrastructure to create cheap imitations of American tech?

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#67

Post by locknitpicker »

vrganj wrote: post #47
HarHarVeryFunny wrote: post #2

Interesting if true - that Moonshot can train a ~3T SOTA model on only 20K NVIDIA GPUs, while others like Musk (who freely admits to distilling OpenAI's models) struggle to build a competitive 1T model (Grok 4.5) with massively more compute (Colossus-1 100-250K GPUs, Colossus-2 500K+ GPUs). I guess at least partly a reflection of all the optimizations in the Kimi 3 architecture. In the recent leaked DeepSeek investor…

> Struggle to build a competitive 1T model (Grok 4.5) with massively more compute (Colossus-1 100-250K GPUs, Colossus-2 500K+ GPUs). This feels like a very American way of designing things - just throw more horse power at it, bigger is better! The rest of the world is usually a bit more resource constrained and efficient at using those resources. See also Mustangs vs German sports cars, giant American fridges, giant…

> This feels like a very American way of designing things - just throw more horse power at it, bigger is better!

I don't think so. Bruteforcing problems is a well established strategy in any field that involves computation of any form. Once you get something working, you can get results right now if you throw resources at it. In the meantime, any improvement in efficiency can easily be back ported to the same computational resources you're using.

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#68

Post by conover »

vrganj wrote: post #47
HarHarVeryFunny wrote: post #2

Interesting if true - that Moonshot can train a ~3T SOTA model on only 20K NVIDIA GPUs, while others like Musk (who freely admits to distilling OpenAI's models) struggle to build a competitive 1T model (Grok 4.5) with massively more compute (Colossus-1 100-250K GPUs, Colossus-2 500K+ GPUs). I guess at least partly a reflection of all the optimizations in the Kimi 3 architecture. In the recent leaked DeepSeek investor…

> Struggle to build a competitive 1T model (Grok 4.5) with massively more compute (Colossus-1 100-250K GPUs, Colossus-2 500K+ GPUs). This feels like a very American way of designing things - just throw more horse power at it, bigger is better! The rest of the world is usually a bit more resource constrained and efficient at using those resources. See also Mustangs vs German sports cars, giant American fridges, giant…

This is the bitter lesson: https://en.wikipedia.org/wiki/Bitter_lesson

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#69

Post by subscribed »

nekusar wrote: post #56
otabdeveloper4 wrote: post #52

Earlier quoted context omitted.

No, it's an attempt to form an artificial moat by buying all the world's GPUs and memory.

True. But check out Ebay right now. Networking gear is CHEAP. I just bought 2 switches, 48 port 10GbE with 4x QSFP+ at 40Gb fiber. $110 each. You can even get 24 port QSFP+ @100Gb networking devices for $350. Yeah while ram and gfx is $$$$$, networking is rock bottom prices..

Try getting something new from Arista, Juniper or Fortigate. Lead times are sad.

(new in terms of the most recent models, not just "something with X number of ports")

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#70

Post by HarHarVeryFunny »

andy_ppp wrote: post #65
UqWBcuFx6NV4r wrote: post #64

Earlier quoted context omitted.

“The only reason that China is good is because it steals from us” spoken from atop the unambiguously crumbling has-been empire. I’m not American nor Chinese, and I’m from a much more US-aligned country. But, Christ, you lot really are asking for it.

I don’t think you can steal someone’s work if it’s based entirely off theft tbh, I see distillation as smart - are you denying it’s a huge part of why Chinese models are competitive or do you believe the narrative yourself that they are doing this with several orders of magnitude less compute and the US labs are profligate and fond of burning money rather than optimising? The truth is probably Anthropic/OpenAI/Google…

The way you make an LLM smarter is by training it on more data, which means you need to make it bigger, which means it costs more to train.

High quality data is expensive. Synthetic data will get you so far, but after that you need to start paying experts to create data for you, which has been going on for a long time. The latest thing is paying for human written LLM-as-judge AI-output evaluation "rubrics", trying to extend RLVR into areas where "looks like it checks the boxes" is the best you can do.

When anyone, Chinese or not (Elon Musk cheerfully admits to distilling OpenAI models) uses the output of someone else's model to train their own, then what they are primarily getting is cheap training data, but you still need to train your model on this data! You may have reduced the cost/speed of training data acquisition, but if you are training a 3T param model (Kimi 3) then you still need the compute to do that - that did not change.

There was an interesting mention of the cost of training data in the recently leaked DeepSeek investor meeting, where their CEO referred to the cost of human-generated training data in China (i.e. using Chinese labor) as being the same as that in the US, which seems surprising. He also mentioned the time such data takes to be created. No doubt the Chinese will catch up in this area - this is just time and money, not Dutch technology (ASML) that the US is blocking them from buying.

Post reply on HN