Live data from Hacker News

Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

tomshardware.com

321–330 of 331 posts

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#321

Earlier quoted context omitted.

So you'd think they'd be able to build a commercial jet liner, no?

So… you'd think the USA would be able to built a nationwide, high-speed rail network? See what I did there?

I mean, they're different industries, so I'm not sure how strong that analogy is.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#322

Earlier quoted context omitted.

If you're one who sees progress as an end goal unto itself, what you describe is a good thing. When one party is attempting novel solutions to outcompete the competition we will be faster to whatever the next change is. That said, I'm not sure what the US policies specifically have to do with this. Countries are always in competition with one another, and if one industry or technology is considered a national securit…

If AI is a threat to other nations, why is anyone even supporting this? Are we really trying to annihilate the planet as quickly as possible?

For the same reason we built nuclear bombs, "if I don't my enemies will"

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#323

Alibaba Cloud claims to reduce Nvidia GPU used for serving unpopular models by 82% (emphasis mine) > 17.7 per cent of GPUs allocated to serve only 1.35 per cent of requests in Alibaba Cloud’s marketplace, the researchers found Instead of 1192 GPUs they now use 213 for serving those requests.

So 82% of 17.7%? 14.5% is worth a raise at least. But it’s still misleading.

I don't think that's what this is saying, isn't it that 100 - ~82 = 17.7% ?

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#324

Earlier quoted context omitted.

So… you'd think the USA would be able to built a nationwide, high-speed rail network? See what I did there?

I mean, they're different industries, so I'm not sure how strong that analogy is.

OP: "So you'd think they'd be able to build a commercial jet liner, no?"

Sigh, the OP chucked out a casual insult at the Chinese because - unlike the USA and EU - their society is unable to (to date) build an international airliner business.

I deftly pointed out that the USA - despite is historical achievements - cannot build a high speed rail network and industry.

Once cannot cherry-pick the data one likes.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#325

Earlier quoted context omitted.

So 82% of 17.7%? 14.5% is worth a raise at least. But it’s still misleading.

I don't think that's what this is saying, isn't it that 100 - ~82 = 17.7% ?

That is a confusing coincidence, but no.

> Reserving full GPU instances for these models leads to allocating 17.7% of our GPUs to serve only 1.35% of requests

> Deployment results show that Aegaeon reduces the number of GPUs required for serving these models from 1,192 to 213, highlighting an 82% GPU resource saving.

82% of their CPUs were serving 98.6% of all traffic. If they reduced the cluster size, they got it to 96.2% of their CPUs serving 98.6% of their traffic. If they reallocated those, which is more likely, then 96.8% of their CPUs are serving 98.6% of all requests, or around 17% more capacity for popular requests on the same hardware.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#326

Earlier quoted context omitted.

There are workarounds and paths forward, but they are definitely being slowed down. I know a lot about this topic.

That sounds like you are moving the goalpost: winning through a narrow definition of "slowing down". Defining "slowing down" as "not having access to western semiconductor ecosystem or 2nm chips" instead of "can they build economically viable AI systems/phones that achieve user objectives satisfactorily, and at scale" does not help your country/bloc with any meaningful outcomes, and merely serves to score Internet po…

No, we have certainly slowed down their progress in AI with export controls on GPUs and upstream. China would have better AI models but for those export controls.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#327

Earlier quoted context omitted.

If AI is a threat to other nations, why is anyone even supporting this? Are we really trying to annihilate the planet as quickly as possible?

For the same reason we built nuclear bombs, "if I don't my enemies will"

AI is a bomb that has already been released.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#328

Earlier quoted context omitted.

If AI is a threat to other nations, why is anyone even supporting this? Are we really trying to annihilate the planet as quickly as possible?

For the same reason we built nuclear bombs, "if I don't my enemies will"

Are we creating enemies so quickly like this?

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#329

Earlier quoted context omitted.

That sounds like you are moving the goalpost: winning through a narrow definition of "slowing down". Defining "slowing down" as "not having access to western semiconductor ecosystem or 2nm chips" instead of "can they build economically viable AI systems/phones that achieve user objectives satisfactorily, and at scale" does not help your country/bloc with any meaningful outcomes, and merely serves to score Internet po…

No, we have certainly slowed down their progress in AI with export controls on GPUs and upstream. China would have better AI models but for those export controls.

"Better" in what way? Ultra-large frontier model performance? China doesn't need that to achieve strategic objectives, like increased productivity and automation, advanced weaponry, advanced manufacturing, user adoption and deployment at scale.

You're narrowing your claim into something you can defend but is strategically hollow. Advanced weaponry, major productivity improvements, R&D speed etc — at the end of day, those are the things the US bloc actually want to slow down. Ultra-large foundational modals was just (incorrectly) seen as the only way to achieve those objectives.

It's like you're arguing that Chinese fighters are inferior to western stealth fighters. That's true when you compare plane-by-plane on paper. And yet the Chinese airfighting system-as-a-whole was still able to down Rafaels with high precision and without being retaliated on, as shown by the India-Pakistan standoff a while ago. What's the point of arguing "we've slowed China's aircraft engine development speed" when they're still shooting down western jets?

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#330

Earlier quoted context omitted.

By perception of the population at least, yes. I mean, the US is literally on the verge of a civil war lol.

Yes, there's a lot of internal division (including now a few assassinations) in the US right now, but in general the US is pretty stable. E.g., it was stable from 1975 to about 2015. The English-speaking lands where the US is now have seen two internal conflicts that killed at least 1% or so of the population: the American revolution, which killed about 1% of the population of the American colonies (but significantly…

1975? how about 1865? there is a real chance we won't ever have free and fair elections again. The amount of people dying isn't relevant to measure stability. political stability and a safe environment for the conduct of commerce is.

Historical stability of China isn't relevant either, I think the modern PRC government of china is all that matters for practical purposes.

Historically, China has been around for like 3 millennia, so it isn't a fair comparison, or a meaningful one.

Post reply on HN