Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

tomshardware.com

Post reply on HN
Print view View original

221–230 of 331 posts

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#221

Post by sokoloff »

heavyset_go wrote: post #189
dlisboa wrote: post #11

Earlier quoted context omitted.

History has shown that withholding technology from China does not significantly stop them and they'll achieve it (or better) in a small number of years. In many senses there's hubris in the western* view of China accomplishments: most of what western companies have created has had significant contribution by Chinese scientists or manufacturing, without which those companies would have nothing. If you look at the name…

It's just straight up low expectations and underestimation derived from racism in the assumption that Americans are smarter and more capable, and Chinese are only good for copying designs and making things we come up with. The idea that they can't do that like we can is pervasive.

It’s absolutely fascinating and mystifying (and dismaying) to see educated and otherwise smart people in engineering in the US hold an opinion that two countries with over a billion people each can’t accomplish anything in engineering or science without “us”.

Despite ample and repeated evidence that they can and, in China’s case, that they’re the best in the world in several areas of manufacturing.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#222

Post by markdown »

ahmeneeroe-v2 wrote: post #160
brookst wrote: post #131

Earlier quoted context omitted.

The US has become unreliable and erratic. Countries aren’t cutting ties or anything, but certainly investing to reduce exposure to capricious US leadership. Much of Europe is increasing domestic military production rather than just buying more from the US precisely because the US has publicly discussed leaving NATO and/or not honoring its guarantees.

Increasing domestic military production is actually a great outcome. Obviously Wall Street would have preferred purchasing from US-listed/owned arms companies, but from the perspective of a military alliance, having well-armed allies is the main point. It's really hard to argue with Trump's methods if they led to Europe finally spending on their own defense.

You've moved the goalposts

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#223

Post by rasz »

segmondy wrote: post #6
hunglee2 wrote: post #2

The US attempt to slow down China's technological development succeeds on the basis of preventing China from directly following the same path, but may backfire in the sense it forces innovation by China in a different direction. The overall outcome for us all may be increase efficiency as a result of this forced innovation, especially if Chinese companies continue to open source their advances, so we may in the end h…

may backfire? it's a bit too late for that. go to 2024, western labs were crushing it. it's now 2025, and from china, we have deepseek, qwen, kimi, glm, ernie and many more capable models keeping up with western labs. there are actually now more chinese labs releasing sota models than western labs.

Have you tried using those models? qwen for example cant even do something as basic as clustering analysis on a list of integers, hell it goes off the rails when just reading said integers from a file - starts babbling about determining number of digits, indexes, tries concatenating numbers together into one big string, no idea wtf is going on with that model.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#224

Post by marcosdumay »

adventured wrote: post #132
ahmeneeroe-v2 wrote: post #124

Earlier quoted context omitted.

Is that the best way? China's rise had already happened by the 2010s Preventing that could have been prevented in the 70s, 80s, 90s by stopping offshoring, blocking student visas, and prosecuting IP theft.

The inclusion of China in the WTO is what changed everything. The elites thought they'd set up shop in a new, gigantic consumer market and reap the rewards. So they got Clinton to spend his last days in office lobbying very aggressively for China's inclusion into the WTO. China had different plans. Keeping the plunderers out (this time) was one of the smartest moves any nation has made in recorded history. Then the s…

> The elites thought they'd set up shop in a new, gigantic consumer market and reap the rewards.

So... Seems that's exactly what they are getting.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#225

Post by NSPG911 »

segmondy wrote: post #6
hunglee2 wrote: post #2

The US attempt to slow down China's technological development succeeds on the basis of preventing China from directly following the same path, but may backfire in the sense it forces innovation by China in a different direction. The overall outcome for us all may be increase efficiency as a result of this forced innovation, especially if Chinese companies continue to open source their advances, so we may in the end h…

may backfire? it's a bit too late for that. go to 2024, western labs were crushing it. it's now 2025, and from china, we have deepseek, qwen, kimi, glm, ernie and many more capable models keeping up with western labs. there are actually now more chinese labs releasing sota models than western labs.

way too early to say that

while qwen, deepseek and kimi are opensourced and good, they are preferred because of their insane token ratio, they use a lot less for more, but a by product is that they are less accurate it is amazing progress by the chinese companies, but they definitely can improve a lot more

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#226

Post by andy_ppp »

svachalek wrote: post #192
bee_rider wrote: post #154

Earlier quoted context omitted.

I’m slightly confuse as to how all this works. Do the GPUs just sit there with the models on them when the models are not in use? I guess I’d assumed this sort of thing would be allocated dynamically. Of course, there’s a benefit to minimizing the number of times you load a model. But surely if a GPU+model is idle for more than a couple minutes it could be freed? (I’m not an AI guy, though—actually I’m used to asking…

Models take a lot of VRAM which is tightly coupled to the GPU so yeah, it's basically sitting there with the model waiting for use. I'm sure they probably do idle out but a few minutes of idle time is a lot of waste--possibly the full 82% mentioned. In this case they optimized by letting the GPUs load multiple models and sharing the load out by token.

How does this work with anything but trivially small context sizes!?

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#227

Post by dotnet00 »

throwacct wrote: post #73
dotnet00 wrote: post #60

Earlier quoted context omitted.

This is such a popular coping tactic from Americans when it comes to facing actual competition, especially from China. Everything they do must either be a lie or just stolen American technology, as if there's something inherently special about Americans that no one else has.

Interesting. So, we're going to deny that most of the IP theft from China up to this moment? Do you even think China is this advanced just because of chinese innovation? C'mon man.

The comment I replied to was dismissing a research paper describing innovation from a chinese company wrt more effective use of their hardware, by insinuating that it must be propaganda with no supporting reasoning.

Of course China has copied foreign technologies, I didn't say they haven't. My point is that you guys love to hang on to that as an excuse to dismiss everything from China even when they're obviously plenty capable of doing R&D in many fields, even with it having gotten its start off "stolen" IP.

America "stole" plenty of rocket technology from Germany, yet it's well understood that they eventually innovated on it and made it their own. But somehow whenever China's involved, you guys come out with your unsubtle bigotry.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#228

Post by skinnymuch »

MSFT_Edging wrote: post #144
dlisboa wrote: post #11

Earlier quoted context omitted.

History has shown that withholding technology from China does not significantly stop them and they'll achieve it (or better) in a small number of years. In many senses there's hubris in the western* view of China accomplishments: most of what western companies have created has had significant contribution by Chinese scientists or manufacturing, without which those companies would have nothing. If you look at the name…

The whole "western" or "the west" always makes me laugh. Half the time it's a dog whistle for "white". Like many right-wing commentators love saying "Western Values" to avoid saying "white, Euro-centric, Christian values". Mexico is a modern country, an industrialized country, a country that is exactly as "western" as the US or Canada. They have the same religious beliefs, speak a dialect of a European language. They…

Mexico isn’t as “western” as the west. Mexico isn’t the first world. It’s not part of the exploiting Global North.

The stuff you bring up ignore the power dynamics which are arguably the most important part.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#229

Post by cindyllm »

dotnet00 wrote: post #227
throwacct wrote: post #73

Earlier quoted context omitted.

Interesting. So, we're going to deny that most of the IP theft from China up to this moment? Do you even think China is this advanced just because of chinese innovation? C'mon man.

The comment I replied to was dismissing a research paper describing innovation from a chinese company wrt more effective use of their hardware, by insinuating that it must be propaganda with no supporting reasoning. Of course China has copied foreign technologies, I didn't say they haven't. My point is that you guys love to hang on to that as an excuse to dismiss everything from China even when they're obviously plen…

[dead]

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#230

Post by hopelite »

lazide wrote: post #145
MSFT_Edging wrote: post #144

Earlier quoted context omitted.

The whole "western" or "the west" always makes me laugh. Half the time it's a dog whistle for "white". Like many right-wing commentators love saying "Western Values" to avoid saying "white, Euro-centric, Christian values". Mexico is a modern country, an industrialized country, a country that is exactly as "western" as the US or Canada. They have the same religious beliefs, speak a dialect of a European language. They…

But why would anyone want to do that? You do realize that antagonizing people with nuclear weapons and the largest economies in the world rarely results in positive results, right?

Arguably the sample size for that is 0 though; but even if we are generous, it’s only about 1 so far.
Post reply on HN