Lots of shareholders here, move along, there is nothing to read
Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system
171–180 of 331 posts
Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system
#172The US attempt to slow down China's technological development succeeds on the basis of preventing China from directly following the same path, but may backfire in the sense it forces innovation by China in a different direction. The overall outcome for us all may be increase efficiency as a result of this forced innovation, especially if Chinese companies continue to open source their advances, so we may in the end h…
It's much easier to copy what others are doing instead of spending the time and money for research and engineering. It's also much easier if you steal the tech. I could never have invented a bicycle but I can sure make a copy of one.
"... instead of spending the time and money for research and engineering..."
China has plenty of R&D and science now.
Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system
#173Earlier quoted context omitted.
> But now the US is trying to … compete against 10x population and lack of similar levels of internal strife and fissures. I can’t tell whether you think the anti-immigration stance is a good thing or bad thing.
it's bad for the US, because China has 10x population. the US can't make up in quality, what it lacks in quantity without immigration and attracting foreigners.
The U.S. actually has more people age 25-64 with a college degree (about 90 million) than China (about 80 million).
Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system
#174Earlier quoted context omitted.
> If for purely political reasons, China may never be able to export these chips to most of the world - which limits their scale - which makes it harder to make them cost effective compared to Western chips. Note that this happens at the same time the US is breaking up its own alliances, so as of this writing, there's no such thing as certainty about politics.
>the same time the US is breaking up its own alliances This isn't happening. The US is driving a harder bargain with our allies. No one serious thinks anyone is walking away from alliances with the US.
Are you sure?
I mean, the current administration has repeatedly threatened to invade militarily two of its allies. Also, it has repeatedly threatened to not honor military agreements with most others, and both the current president and vice-president have insulted the leaders of several allied countries to their face.
Oh, and if that weren't sufficient, the current admin has unilaterally broken all trade treaties (alongside most intellectual property treaties) it held with its commercial partners.
The EU is slow at it, but it's no accident that everybody is doing their best to move away from US tech and military dependencies.
Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system
#175Earlier quoted context omitted.
Why the framing of alliances like it’s a boolean? The question of “can we trust the American government” is now being asked more often. Existing alliances and new potential alliances face that question, whether or not you personally believe that they should trust America. Even if no concrete actions are being performed with asking that question, the fact that question is even being asked is a major drop from where we…
You're right, alliances are not boolean. From the US perspective, we have been asking ourselves "can we trust Europe's military capacity" for a very long time and the answer (prior to 2025) was: NO. With Trump on one side and Russia on the other, it seems like the answer has shifted to: MAYBE.
Frankly, this sounds like you're repeating propaganda.
When the US called its allies to its wars, NATO responded. Now that the rest of NATO is being threatened, the US is playing neutral, trying to see which side will bid highest for their help.
Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system
#176Earlier quoted context omitted.
History has shown that withholding technology from China does not significantly stop them and they'll achieve it (or better) in a small number of years. In many senses there's hubris in the western* view of China accomplishments: most of what western companies have created has had significant contribution by Chinese scientists or manufacturing, without which those companies would have nothing. If you look at the name…
> History has shown that withholding technology from China does not significantly stop them and they'll achieve it (or better) in a small number of years. It's worked for a very long time for aircraft. China has been pushing to build its own aircraft for >23 years. It took 14 years for COMAC to get its first regional jet flying commercial flights on a Chinese airline, and 21 years to get a narrow-body plane flying a…
> China may never be able to export these chips
While you type this, the rest of the world is already using Chinese cars, something that was unthinkable a year or two ago.
The US has closed the market off from this for its auto industry to survive.
Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system
#177Earlier quoted context omitted.
>the same time the US is breaking up its own alliances This isn't happening. The US is driving a harder bargain with our allies. No one serious thinks anyone is walking away from alliances with the US.
No one is going to walk away from that kind of alliance tomorrow, sure. Stuff like "we're going to remotely disable military equipment we've sold you" is going to have consequences though. It's not walking away from alliances, it's just focusing on more stable countries.
To clarify: it's not exactly "remotely disable".
It's "block everything that depends on US clouds", which is a considerable downgrade (because you can't upload all mission parameters to an airplane without going through the cloud, and you can't use self-diagnosis features), but not entirely a kill switch. Close enough, though.
Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system
#178Earlier quoted context omitted.
>"...The issue is allowing the Chinese to steal IP..." I do not think they need permission. There is no force that could order country to recognize IP. Do you really expect all world forever pay rent to few giant corps?
Bad reading comprehension. You're talking about recognizing IP, I am talking about stealing IP AND selling stolen IP in our markets. 1st: yes force can be used to discourage the theft of IP. This is merely an obstacle, not a total blocker 2nd: yes force can be used to block IP from our markets. This is actually incredibly trivial and would have been very easy 40-years ago.
>You're talking about recognizing IP, I am talking about stealing IP ..."
If country does not recognize IP then "stealing" is not a theft in their eyes.
As for using force to prevent "theft": what force? Military? You might get burned really bad.
Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system
#179Alibaba Cloud claims to reduce Nvidia GPU used for serving unpopular models by 82% (emphasis mine) > 17.7 per cent of GPUs allocated to serve only 1.35 per cent of requests in Alibaba Cloud’s marketplace, the researchers found Instead of 1192 GPUs they now use 213 for serving those requests.
I’m slightly confuse as to how all this works. Do the GPUs just sit there with the models on them when the models are not in use? I guess I’d assumed this sort of thing would be allocated dynamically. Of course, there’s a benefit to minimizing the number of times you load a model. But surely if a GPU+model is idle for more than a couple minutes it could be freed? (I’m not an AI guy, though—actually I’m used to asking…
The paper is about techniques to do that dynamic allocation to maximize utilization without incurring unacceptable latencies. If you let a GPU sit idle for several minutes after serving a single request, you're setting money on fire. So they reuse it for a different model as soon as possible, starting even before the first request is finished, because: If you don't have a dedicated GPU for a model, are you going to wait for a multi-gigabyte transfer before each request? So they have a dedicated GPU (or two, one for prefill, one for decode) for a group of models that are processed in an interleaved fashion, scheduled such that they stay within the latency budget.
Re: Alibaba Cloud says it cut Nvidia AI GPU use by 82% with new pooling system
#180Earlier quoted context omitted.
Why the framing of alliances like it’s a boolean? The question of “can we trust the American government” is now being asked more often. Existing alliances and new potential alliances face that question, whether or not you personally believe that they should trust America. Even if no concrete actions are being performed with asking that question, the fact that question is even being asked is a major drop from where we…
You're right, alliances are not boolean. From the US perspective, we have been asking ourselves "can we trust Europe's military capacity" for a very long time and the answer (prior to 2025) was: NO. With Trump on one side and Russia on the other, it seems like the answer has shifted to: MAYBE.
> From the US perspective, we have been asking ourselves "can we trust Europe's military capacity" for a very long time and the answer (prior to 2025) was: NO.
NATO's mutual defense clause has only been activated once: after 9/11, when the United States declared war on the Taliban in Afghanistan.
Out of the 3621 deaths of coalition soldiers, 1160 of them were from nations other than the United States, including 457 from the UK, 159 from Canada, and 90 from France.
https://en.wikipedia.org/wiki/Coalition_casualties_in_Afghan...