Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

tomshardware.com

Post reply on HN
Print view View original

61–70 of 331 posts

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#61

Post by tsunamifury »

reliabilityguy wrote: post #5
hunglee2 wrote: post #2

The US attempt to slow down China's technological development succeeds on the basis of preventing China from directly following the same path, but may backfire in the sense it forces innovation by China in a different direction. The overall outcome for us all may be increase efficiency as a result of this forced innovation, especially if Chinese companies continue to open source their advances, so we may in the end h…

Tbh this whole situation reminds of how Japan excelled in making a lot more with a lot less after WW2, e.g., fuel-efficient engines, light cars, etc. these constraints were not present in the US (and to some extent in Europe), and resulted in US cars being completely not competitive in non-US markets.

The premature optimizer is never the innovator.

Japan eventually stopped that role and their products improved greatly.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#62

Post by currydove »

myth_drannon wrote: post #57
Hikikomori wrote: post #50

Earlier quoted context omitted.

And the US has never stolen IP?

Corporate espionage is ever present but it is criminalized. The only time US as a country did that you can say "stole IP" was after WII when it took Nazi rocket scientists and technology. China is the opposite; stealing tech is done by the state apparatus (same was done by USSR and reverse engineering computers for example). Frankly I'm not surprised that this is done, probably if US was so behind it would have done…

Negative.

https://interestingengineering.com/innovation/the-spies-who-...

https://yawboadu.substack.com/p/stealing-success-how-ip-thef...

https://ipwatchdog.com/2017/07/05/americas-industrial-revolu...

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#63

Post by yorwba »

kilotaras wrote: post #47

Alibaba Cloud claims to reduce Nvidia GPU used for serving unpopular models by 82% (emphasis mine) > 17.7 per cent of GPUs allocated to serve only 1.35 per cent of requests in Alibaba Cloud’s marketplace, the researchers found Instead of 1192 GPUs they now use 213 for serving those requests.

Not really, Figure 1(a) of the paper says that the 17.7% are relative to a total of 30k GPUs (i.e. 5310 GPUs for handling those 1.35% of requests) and the reduction is measured in a smaller beta deployment with only 47 different models (vs. the 733 "cold" models overall.) Naïve extrapolation by model count suggests they would need 3321 GPUs to serve all cold models, a 37.5% reduction to before. (Or 6.6% reduction of the full 30k-GPU cluster.)

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#64

Post by g8oz »

myth_drannon wrote: post #57
Hikikomori wrote: post #50

Earlier quoted context omitted.

And the US has never stolen IP?

Corporate espionage is ever present but it is criminalized. The only time US as a country did that you can say "stole IP" was after WII when it took Nazi rocket scientists and technology. China is the opposite; stealing tech is done by the state apparatus (same was done by USSR and reverse engineering computers for example). Frankly I'm not surprised that this is done, probably if US was so behind it would have done…

You are mistaken about American intellectual property theft. They engaged in extensive IP theft from Britain in the 18th and 19th century with the encouragement of the government. See https://interestingengineering.com/innovation/the-spies-who-...

Also during World War I the American government seized German chemical patents thereby launching the American chemical industry. So that is an example of theft by the state apparatus.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#65

Post by rayiner »

notepad0x90 wrote: post #12
hunglee2 wrote: post #2

The US attempt to slow down China's technological development succeeds on the basis of preventing China from directly following the same path, but may backfire in the sense it forces innovation by China in a different direction. The overall outcome for us all may be increase efficiency as a result of this forced innovation, especially if Chinese companies continue to open source their advances, so we may in the end h…

I think anti-immigrant rhetoric will have the most impact against the US. A lot of the people innovating on this stuff are being maligned and leaving in droves. Aside from geography, attracting talent from all over the world is the one edge the US has a nation over countries like China. But now the US is trying to be xenophobic like China, restrict tech import/export like China but compete against 10x population and…

> But now the US is trying to … compete against 10x population and lack of similar levels of internal strife and fissures.

I can’t tell whether you think the anti-immigration stance is a good thing or bad thing.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#66

Post by lossolo »

myth_drannon wrote: post #42
hunglee2 wrote: post #2

The US attempt to slow down China's technological development succeeds on the basis of preventing China from directly following the same path, but may backfire in the sense it forces innovation by China in a different direction. The overall outcome for us all may be increase efficiency as a result of this forced innovation, especially if Chinese companies continue to open source their advances, so we may in the end h…

China's innovation relies on the stolen western IP, without it, China is nothing. Also USSR/Russia is no longer a scientific powerhouse that can supply China with some military innovation. A dictatorship combined with cheap labour it 100% guarantees that the country's innovation is stunted, no matter what the Chinese propaganda claims.

Every single sentence you wrote is untrue and can be disproven by empirical evidence. You can learn about it here[1][2].

1. https://itif.org/publications/2024/09/16/china-is-rapidly-be...

2. https://www.economist.com/science-and-technology/2024/06/12/...

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#67

Post by lossolo »

dotnet00 wrote: post #60
muddi900 wrote: post #32

[flagged]

This is such a popular coping tactic from Americans when it comes to facing actual competition, especially from China. Everything they do must either be a lie or just stolen American technology, as if there's something inherently special about Americans that no one else has.

It comes from what people are taught in schools and from their own self perception. When those beliefs about American exceptionalism are challenged, cognitive dissonance kicks in.

https://en.wikipedia.org/wiki/American_exceptionalism

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#68

Post by ikidd »

dlisboa wrote: post #21
notepad0x90 wrote: post #12

Earlier quoted context omitted.

I think anti-immigrant rhetoric will have the most impact against the US. A lot of the people innovating on this stuff are being maligned and leaving in droves. Aside from geography, attracting talent from all over the world is the one edge the US has a nation over countries like China. But now the US is trying to be xenophobic like China, restrict tech import/export like China but compete against 10x population and…

> But, China's greatest weakness is their lack of ambition and focus on regional matters like Taiwan and south china sea, instead of winning over western europe and india. That's a strength. Them not having interest in global domination and regime change other than their backyard is what allows them to easily make partners in Africa and LATAM, the most important regions for raw materials.

>Them not having interest in global domination and regime change

I don't even know where to begin with that one.

Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system

#69

Post by archerx »

hunglee2 wrote: post #2

The US attempt to slow down China's technological development succeeds on the basis of preventing China from directly following the same path, but may backfire in the sense it forces innovation by China in a different direction. The overall outcome for us all may be increase efficiency as a result of this forced innovation, especially if Chinese companies continue to open source their advances, so we may in the end h…

I want China to release GPUs with a ton of VRAM, 128gb - 256gb. It doesn’t matter if they are half as fast as Nvidia because having a big model at a reasonable speed is better than not being to run them at all. AMD could have done this and have had a massive impact on nvidia’s market share but they choose not to because reasons.
Post reply on HN