Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

Post by skinnymuch » Tue, Oct 21, 2025, 12:11 AM UTC

OrvalWintermute wrote: post #35

dlisboa wrote: post #21

Earlier quoted context omitted.

> But, China's greatest weakness is their lack of ambition and focus on regional matters like Taiwan and south china sea, instead of winning over western europe and india. That's a strength. Them not having interest in global domination and regime change other than their backyard is what allows them to easily make partners in Africa and LATAM, the most important regions for raw materials.

if you've been tracking the shark deals they give countries for loans, I think you'd recant what you just said. "while the CCP accuses the West of predatory interest rates, the average Chinese rescue loan carries an interest rate of about 5 percent, more than double the IMF’s standard 2 percent. As of Oct. 1, 2025, despite higher U.S. interest rates, the IMF’s Special Drawing Rights lending rate stands at only 3.41 p…

Post by lazide » Tue, Oct 21, 2025, 12:17 AM UTC

hopelite wrote: post #230

lazide wrote: post #145

Earlier quoted context omitted.

But why would anyone want to do that? You do realize that antagonizing people with nuclear weapons and the largest economies in the world rarely results in positive results, right?

Arguably the sample size for that is 0 though; but even if we are generous, it’s only about 1 so far.

Post by jychang » Tue, Oct 21, 2025, 12:39 AM UTC

svachalek wrote: post #192

bee_rider wrote: post #154

Earlier quoted context omitted.

I’m slightly confuse as to how all this works. Do the GPUs just sit there with the models on them when the models are not in use? I guess I’d assumed this sort of thing would be allocated dynamically. Of course, there’s a benefit to minimizing the number of times you load a model. But surely if a GPU+model is idle for more than a couple minutes it could be freed? (I’m not an AI guy, though—actually I’m used to asking…

Models take a lot of VRAM which is tightly coupled to the GPU so yeah, it's basically sitting there with the model waiting for use. I'm sure they probably do idle out but a few minutes of idle time is a lot of waste--possibly the full 82% mentioned. In this case they optimized by letting the GPUs load multiple models and sharing the load out by token.

Post by [deleted] » Tue, Oct 21, 2025, 12:45 AM UTC

rayiner wrote: post #65

notepad0x90 wrote: post #12

Earlier quoted context omitted.

I think anti-immigrant rhetoric will have the most impact against the US. A lot of the people innovating on this stuff are being maligned and leaving in droves. Aside from geography, attracting talent from all over the world is the one edge the US has a nation over countries like China. But now the US is trying to be xenophobic like China, restrict tech import/export like China but compete against 10x population and…

> But now the US is trying to … compete against 10x population and lack of similar levels of internal strife and fissures. I can’t tell whether you think the anti-immigration stance is a good thing or bad thing.

Post by hollerith » Tue, Oct 21, 2025, 12:49 AM UTC

notepad0x90 wrote: post #143

rayiner wrote: post #65

Earlier quoted context omitted.

> But now the US is trying to … compete against 10x population and lack of similar levels of internal strife and fissures. I can’t tell whether you think the anti-immigration stance is a good thing or bad thing.

it's bad for the US, because China has 10x population. the US can't make up in quality, what it lacks in quantity without immigration and attracting foreigners.