Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

tomshardware.com

Post reply on HN
Print view View original

151–160 of 331 posts

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#151

Post by Jackpillar »

ikidd wrote: post #68
dlisboa wrote: post #21

Earlier quoted context omitted.

> But, China's greatest weakness is their lack of ambition and focus on regional matters like Taiwan and south china sea, instead of winning over western europe and india. That's a strength. Them not having interest in global domination and regime change other than their backyard is what allows them to easily make partners in Africa and LATAM, the most important regions for raw materials.

>Them not having interest in global domination and regime change I don't even know where to begin with that one.

Yeah waiting to see historical examples of contemporary China being interested in global domination and regime change, especially in contrast to the US.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#152

Post by deadbabe »

hshdhdhj4444 wrote: post #110
dlisboa wrote: post #11

Earlier quoted context omitted.

History has shown that withholding technology from China does not significantly stop them and they'll achieve it (or better) in a small number of years. In many senses there's hubris in the western* view of China accomplishments: most of what western companies have created has had significant contribution by Chinese scientists or manufacturing, without which those companies would have nothing. If you look at the name…

Ironically, the best way America could have prevented China’s rise in tech was by stapling green cards to diplomas of Chinese citizens who completed their higher education in the U.S. like the plan in the early 2010s.

But they didn’t do it, because the current administration can’t get it through their thick skulls that the key advantage the US can have in this world is a monopoly on all the really smart people.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#153

Post by FpUser »

ahmeneeroe-v2 wrote: post #133
brookst wrote: post #129

Earlier quoted context omitted.

Those students would have just gone to other countries, written their PhD dissertations there, advanced another country’s tech sector, and the US would have found the pain of isolationism that much sooner. It is not possible to keep core IP secret. HN folks, of all people, should know this. Anything that thousands of people know is de facto public knowledge.

Keeping core IP secret is not the issue. The issue is allowing the Chinese to steal IP and then compete in our markets with that stolen IP. >students would have just gone to other countries, written their PhD dissertations there, advanced another country’s tech sector, which other countries specifically? No other country has a tech sector. It's the US hegemony or the China hegemony.

>"...The issue is allowing the Chinese to steal IP..."

I do not think they need permission. There is no force that could order country to recognize IP. Do you really expect all world forever pay rent to few giant corps?

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#154

Post by bee_rider »

kilotaras wrote: post #47

Alibaba Cloud claims to reduce Nvidia GPU used for serving unpopular models by 82% (emphasis mine) > 17.7 per cent of GPUs allocated to serve only 1.35 per cent of requests in Alibaba Cloud’s marketplace, the researchers found Instead of 1192 GPUs they now use 213 for serving those requests.

I’m slightly confuse as to how all this works. Do the GPUs just sit there with the models on them when the models are not in use?

I guess I’d assumed this sort of thing would be allocated dynamically. Of course, there’s a benefit to minimizing the number of times you load a model. But surely if a GPU+model is idle for more than a couple minutes it could be freed?

(I’m not an AI guy, though—actually I’m used to asking SLURM for new nodes with every run I do!)

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#155

Post by traverseda »

ahmeneeroe-v2 wrote: post #127
Yoric wrote: post #41

Earlier quoted context omitted.

> If for purely political reasons, China may never be able to export these chips to most of the world - which limits their scale - which makes it harder to make them cost effective compared to Western chips. Note that this happens at the same time the US is breaking up its own alliances, so as of this writing, there's no such thing as certainty about politics.

>the same time the US is breaking up its own alliances This isn't happening. The US is driving a harder bargain with our allies. No one serious thinks anyone is walking away from alliances with the US.

No one is going to walk away from that kind of alliance tomorrow, sure. Stuff like "we're going to remotely disable military equipment we've sold you" is going to have consequences though. It's not walking away from alliances, it's just focusing on more stable countries.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#156

Post by lurk2 »

hshdhdhj4444 wrote: post #110
dlisboa wrote: post #11

Earlier quoted context omitted.

History has shown that withholding technology from China does not significantly stop them and they'll achieve it (or better) in a small number of years. In many senses there's hubris in the western* view of China accomplishments: most of what western companies have created has had significant contribution by Chinese scientists or manufacturing, without which those companies would have nothing. If you look at the name…

Ironically, the best way America could have prevented China’s rise in tech was by stapling green cards to diplomas of Chinese citizens who completed their higher education in the U.S. like the plan in the early 2010s.

Massive vector for theft of trade secrets and intellectual property.

It’s notable that China did not adopt the same policy during the period you are associating with their rise. Indeed, they’ve taken the opposite stance in recent years and (now that they have stolen American IP) have moved to seize control of assets and expel the superfluous foreigners.

There is a lesson to be learned there, but it’s contrary to the argument you are trying to make.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#157

Post by ahmeneeroe-v2 »

delfinom wrote: post #130
ahmeneeroe-v2 wrote: post #124

Earlier quoted context omitted.

Is that the best way? China's rise had already happened by the 2010s Preventing that could have been prevented in the 70s, 80s, 90s by stopping offshoring, blocking student visas, and prosecuting IP theft.

Eh? They are a country with a culture that values education like a few other asian countries. They were set back by the colonial bullshit of Europe, devastation of WW2 and communist revolutions and so. They would have gotten to the same point as they are today, just longer if anything if any attempt was made to hamper it.

You're 100% correct. "Prevented" is too strong of a word (that I borrowed from GP).

I would characterize my recommendations as things that could have been done for the US to not fund or encourage the re-rise of China.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#158

Post by ahmeneeroe-v2 »

FpUser wrote: post #153
ahmeneeroe-v2 wrote: post #133

Earlier quoted context omitted.

Keeping core IP secret is not the issue. The issue is allowing the Chinese to steal IP and then compete in our markets with that stolen IP. >students would have just gone to other countries, written their PhD dissertations there, advanced another country’s tech sector, which other countries specifically? No other country has a tech sector. It's the US hegemony or the China hegemony.

>"...The issue is allowing the Chinese to steal IP..." I do not think they need permission. There is no force that could order country to recognize IP. Do you really expect all world forever pay rent to few giant corps?

Bad reading comprehension.

You're talking about recognizing IP, I am talking about stealing IP AND selling stolen IP in our markets.

1st: yes force can be used to discourage the theft of IP. This is merely an obstacle, not a total blocker 2nd: yes force can be used to block IP from our markets. This is actually incredibly trivial and would have been very easy 40-years ago.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#159

Post by make3 »

bee_rider wrote: post #154
kilotaras wrote: post #47

Alibaba Cloud claims to reduce Nvidia GPU used for serving unpopular models by 82% (emphasis mine) > 17.7 per cent of GPUs allocated to serve only 1.35 per cent of requests in Alibaba Cloud’s marketplace, the researchers found Instead of 1192 GPUs they now use 213 for serving those requests.

I’m slightly confuse as to how all this works. Do the GPUs just sit there with the models on them when the models are not in use? I guess I’d assumed this sort of thing would be allocated dynamically. Of course, there’s a benefit to minimizing the number of times you load a model. But surely if a GPU+model is idle for more than a couple minutes it could be freed? (I’m not an AI guy, though—actually I’m used to asking…

the models are huge, so not a single (latest gen) one can fit on a single GPU.

It's likely that these are small unpopular (non flagship) models, or that they only pack eg one layer of each model.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#160

Post by ahmeneeroe-v2 »

brookst wrote: post #131
ahmeneeroe-v2 wrote: post #127

Earlier quoted context omitted.

>the same time the US is breaking up its own alliances This isn't happening. The US is driving a harder bargain with our allies. No one serious thinks anyone is walking away from alliances with the US.

The US has become unreliable and erratic. Countries aren’t cutting ties or anything, but certainly investing to reduce exposure to capricious US leadership. Much of Europe is increasing domestic military production rather than just buying more from the US precisely because the US has publicly discussed leaving NATO and/or not honoring its guarantees.

Increasing domestic military production is actually a great outcome.

Obviously Wall Street would have preferred purchasing from US-listed/owned arms companies, but from the perspective of a military alliance, having well-armed allies is the main point.

It's really hard to argue with Trump's methods if they led to Europe finally spending on their own defense.

Post reply on HN