Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

bloomberg.com

Post reply on HN
Print view View original

71–79 of 79 posts

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#71

Post by fellowniusmonk »

blazarquasar wrote: post #42
fellowniusmonk wrote: post #22

It's also built on the back of downloading all the content of the top ten sites on the list, the fact that all the independent content sites with their "pre-war" steel haven't been acquired at top dollar by U.S. companies shows the existential threat / stop China posturing is a total joke. https://archive.is/AvWWX

No one is using that dataset for any serious models. It’s way too small and old.

Tell that to the nation states that are actively trying to find every exploit they can to pull it down.

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#72

Post by hereme888 »

hereme888 wrote: post #66
UqWBcuFx6NV4r wrote: post #64

Earlier quoted context omitted.

“The only reason that China is good is because it steals from us” spoken from atop the unambiguously crumbling has-been empire. I’m not American nor Chinese, and I’m from a much more US-aligned country. But, Christ, you lot really are asking for it.

Phew, if that's not clearly anti-American hate, insult towards Christianity, hatred in general.... Then idk what is. And btw, yes it's an objective fact from every technical angle that the Chinese only have competitive models because of US tech. Why do you think they try so hard to smuggle NVIDIA GPUs, and now exposed infrastructure to create cheap imitations of American tech?

[dead]

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#73

Post by theblazehen »

infecto wrote: post #46
upbeat_general wrote: post #33

Earlier quoted context omitted.

This is just not how it works - it is perfectly plausible (in fact the most likely) that pretraining was well finished by the time Fable was released. A bit of extra distillation takes far less compute. Moreover, it’s very plausible (and expected) to use multiple clusters and GPU types for RL rollouts which could very well not be included in this count. No part of this pipeline is fixed in stone.

Relatedly I also wonder how much the orchestrated distillation bot nets that Anthropic uses against China are real. Not saying it does not happen but I have never seen dating showing how pervasive it is and I always imagine other western labs would absolutely be doing the same.

There's significant amounts of traffic going through transfer stations [1], which is then re-sold to the model labs. For a while the DeepSeek API was serving Fable as the DS 4 Pro model even. [2]

[1] https://www.chinatalk.media/p/how-to-buy-cheap-claude-tokens... [2] https://xcancel.com/synthwavedd/status/2078514339552628880

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#74

Post by HarHarVeryFunny »

hereme888 wrote: post #66
UqWBcuFx6NV4r wrote: post #64

Earlier quoted context omitted.

“The only reason that China is good is because it steals from us” spoken from atop the unambiguously crumbling has-been empire. I’m not American nor Chinese, and I’m from a much more US-aligned country. But, Christ, you lot really are asking for it.

Phew, if that's not clearly anti-American hate, insult towards Christianity, hatred in general.... Then idk what is. And btw, yes it's an objective fact from every technical angle that the Chinese only have competitive models because of US tech. Why do you think they try so hard to smuggle NVIDIA GPUs, and now exposed infrastructure to create cheap imitations of American tech?

Just in case you are interested in any facts ...

The Trump administration, having first blocked China from buying NVIDIA H100's, has since done a U-turn and is now allowing them to buy the more powerful NVIDIA H200, on a case-by-case basis.

Now, the CHINESE government is blocking Chinese companies from buying these H200s, at least in part because it turns out that being denied US tech has been a great accelerator for Chinese tech, with Huawei now producing the entirely domestic Ascend 950 chip, which according to NVIDIA's Jensen Huang performs about the same as NVIDIA's own H100 (which while not NVIDIA's most powerful is still plenty capable, and is what Elon Musk's Colossus-1 data center mostly uses, currently being rented out to Anthropic).

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#75

Post by hereme888 »

HarHarVeryFunny wrote: post #74
hereme888 wrote: post #66

Earlier quoted context omitted.

Phew, if that's not clearly anti-American hate, insult towards Christianity, hatred in general.... Then idk what is. And btw, yes it's an objective fact from every technical angle that the Chinese only have competitive models because of US tech. Why do you think they try so hard to smuggle NVIDIA GPUs, and now exposed infrastructure to create cheap imitations of American tech?

Just in case you are interested in any facts ... The Trump administration, having first blocked China from buying NVIDIA H100's, has since done a U-turn and is now allowing them to buy the more powerful NVIDIA H200, on a case-by-case basis. Now, the CHINESE government is blocking Chinese companies from buying these H200s, at least in part because it turns out that being denied US tech has been a great accelerator for…

H100 export licensing restrictions began on August 26, 2022, under the Biden administration.

The CCP tried to promote their own chips, but later regressed and started allowing ByteDance, Alibaba, and Tencent to purchase more than 400,000 H200s. And they are trying to acquire millions.

Thus, export restrictions accelerated Chinese substitution, but also made that substitution slower, costlier, less scalable, and technically inferior to unrestricted Nvidia access.

> According to Jensen Huang, Ascend 950 performs about the same as H100

Source?

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#76

Post by sirsinsalot »

nomel wrote: post #59
corranh wrote: post #21

Earlier quoted context omitted.

I’m still baffled that OpenAI can complain about this with a straight face while these models are literally trained on everything regardless of copyright or permission.

Raw stolen data that is in no way related to AI vs a very very expensive transformative compilation of that raw stolen data that results in usable AI. The frustration is completely understandable to me, since distillation skips much of that "very very expensive" part. And yes, I understand the stolen data was expensive to make, so I understand the owners of it are also frustrated, but that's partly a problem with cur…

You're being reductive. It isn't just one copy of the book. It is using the book to enable, in some part, value to be generated from it that goes in the pocket of a rent seeker (AI business) and not the author.

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#77

Post by bigyabai »

hereme888 wrote: post #75
HarHarVeryFunny wrote: post #74

Earlier quoted context omitted.

Just in case you are interested in any facts ... The Trump administration, having first blocked China from buying NVIDIA H100's, has since done a U-turn and is now allowing them to buy the more powerful NVIDIA H200, on a case-by-case basis. Now, the CHINESE government is blocking Chinese companies from buying these H200s, at least in part because it turns out that being denied US tech has been a great accelerator for…

H100 export licensing restrictions began on August 26, 2022, under the Biden administration. The CCP tried to promote their own chips, but later regressed and started allowing ByteDance, Alibaba, and Tencent to purchase more than 400,000 H200s. And they are trying to acquire millions. Thus, export restrictions accelerated Chinese substitution, but also made that substitution slower, costlier, less scalable, and techn…

It's been great. I hope they keep importing them, Qwen and Z.ai have done far more for humanity with that compute than Anthropic or OpenAI ever did.

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#78

Post by nomel »

sirsinsalot wrote: post #76
nomel wrote: post #59

Earlier quoted context omitted.

Raw stolen data that is in no way related to AI vs a very very expensive transformative compilation of that raw stolen data that results in usable AI. The frustration is completely understandable to me, since distillation skips much of that "very very expensive" part. And yes, I understand the stolen data was expensive to make, so I understand the owners of it are also frustrated, but that's partly a problem with cur…

You're being reductive. It isn't just one copy of the book. It is using the book to enable, in some part, value to be generated from it that goes in the pocket of a rent seeker (AI business) and not the author.

I understand what you're saying, but give my comment a second read. It doesn't matter what me or you think, if they current law doesn't allow the content creators to demand compensation.

Re: Moonshot’s Kimi uses 20k Nvidia chip cluster from Alibaba

#79

Post by nomel »

HarHarVeryFunny wrote: post #61
nomel wrote: post #59

Earlier quoted context omitted.

Raw stolen data that is in no way related to AI vs a very very expensive transformative compilation of that raw stolen data that results in usable AI. The frustration is completely understandable to me, since distillation skips much of that "very very expensive" part. And yes, I understand the stolen data was expensive to make, so I understand the owners of it are also frustrated, but that's partly a problem with cur…

Not really, in this context the word "distillation" is really being abused, or at least used in a different sense than when it was originally introduced in the Hinton et al "Distilling the Knowledge in a Neural Network" paper, where it was essentially referring to knowledge compression. The way Anthropic are using "distillation" is just in a very broad vague sense to claim that some data generated by their model was…

I assume that we're all familiar with the concept of distillation here, so didn't feel the need to point out its current misuse towards models that aren't smaller. In all the cases with China, the models are smaller, so it still correctly applies.

> but unlike Anthropic they at least didn't break copyright law

Reference? All the recent lawsuits have sides with Anthropic, that I've seen. What exact case do you have in mind?

I am, clearly, not talking about the act of not paying to obtain the works. That is wrong, and they've been sued successfully, because the law is clear about that. I'm talking about the integration of works into the model, without permission, which everything I've seen says none is required in a "sufficiently transformative" framing.

Post reply on HN