Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CUDA Moat Still Alive

semianalysis.com

Post reply on HN
Print view View original

121–130 of 176 posts

Re: CUDA Moat Still Alive

#121

Post by imtringued »

croes wrote: post #95
ryao wrote: post #54

> AMD is attempting to vertically integrate next year with their upcoming Pollara 400G NIC, which supports Ultra Ethernet, hopefully making AMD competitive with Nvidia. Infiniband is an industry standard. It is weird to see the industry invent yet another standard to do effectively the same thing just because Nvidia is using it. This “Nvidia does things this way so let’s do it differently” mentality is hurting AMD: *…

You hardly beat someone by copying him. They have way more experience in the field you try to catch up.

AMD doesn't need to beat Nvidia, they just need to match them at a lower price point.

Re: CUDA Moat Still Alive

#122

Post by pjmlp »

Comma2976 wrote: post #118
pjmlp wrote: post #110

Earlier quoted context omitted.

It would go nowhere, games history is full of great hardware that died because they failed to have a profitable ecosystem. Even Steam Deck is only a success, because it depends on Windows ecosystem, and the moment Microsoft decides it is enough, lets see how long it holds.

Steam Decks run on Arch Linux

As means to avoid paying for Windows licenses.

All the games that matter are Windows games running via Proton, as Valve has failed to actually build a GNU/Linux native games ecosystem, in spite of UNIX/POSIX underpinnings of Android NDK, PlayStation, the studios hardly bother.

The day Microsoft actually decides to challenge Proton, or do a netbooks move on handhelds with XBox OS/Windows, the SteamDeck will lose, just like the netboooks did.

Additionally, it is anyone's guess what will happen to Valve when Gabe steps down.

Re: CUDA Moat Still Alive

#123

Post by imtringued »

kevingadd wrote: post #36
Lerc wrote: post #32

That MatMul performance is fairly shocking. To be that much below theoretical maximum on what should be a fairly low overhead operation. I would at least hope that they know where the speed is going, but the issue of torch.matmul and F.Linear using different libraries with different performance suggests that they don't even know which code they are running, let alone where the slow bits in that code are.

Low overhead in what sense? matmul is kinda complicated and there are varying, complex state-of-the-art algorithms for it, no? And then if you know things about the matrices in advance you can start optimizing for that, which adds another layer of complexity.

Matmul is trivial to get right, especially since you won't be calculating dot products manually to begin with. You're going to use the tensor cores or equivalent, which already perform almost the entire matrix multiplication for you. Your primary goal in developing a custom matmul kernel is in adjusting the algorithm to the specific hardware by knowing how many tiles you can store in your local registers and SRAM and how to simultaneously intertwine loading new data from HBM and performing the calculations.

Re: CUDA Moat Still Alive

#124

Post by almostgotcaught »

sangnoir wrote: post #111
Twirrim wrote: post #84

Earlier quoted context omitted.

Engineers at hyperscalers are struggling through all the bugs too. It's coming at notable opportunity cost for them, at a time when they also want an end to the monopoly. Do they buy AMD and wade through bug after bug, regression after regression, or do they shell out slightly more money for Nvidia GPUs and have it "just work". AMD has to get on top of their software quality issues if they're ever going to succeed in…

> Engineers at hyperscalers are struggling through all the bugs too [citation needed]

Lolol 100% accurate. Go trawl through PRs to Triton by FB people to the AMD portion of the codebase.

Re: CUDA Moat Still Alive

#125

Post by Twirrim »

sangnoir wrote: post #111
Twirrim wrote: post #84

Earlier quoted context omitted.

Engineers at hyperscalers are struggling through all the bugs too. It's coming at notable opportunity cost for them, at a time when they also want an end to the monopoly. Do they buy AMD and wade through bug after bug, regression after regression, or do they shell out slightly more money for Nvidia GPUs and have it "just work". AMD has to get on top of their software quality issues if they're ever going to succeed in…

> Engineers at hyperscalers are struggling through all the bugs too [citation needed]

Sorry, but NDAs mean I can't say any more.

Re: CUDA Moat Still Alive

#126

Post by iszomer »

mixblast wrote: post #120
ryao wrote: post #54

> AMD is attempting to vertically integrate next year with their upcoming Pollara 400G NIC, which supports Ultra Ethernet, hopefully making AMD competitive with Nvidia. Infiniband is an industry standard. It is weird to see the industry invent yet another standard to do effectively the same thing just because Nvidia is using it. This “Nvidia does things this way so let’s do it differently” mentality is hurting AMD: *…

Infiniband is a monopoly from NVidia (Mellanox). Everyone else would much rather use Ethernet which is the actual industry standard.

Depends on deployment: Ethernet has more drawbacks encapsulating PCIe packet traffic than Infiniband does, or lack thereof with RDMA.

Re: CUDA Moat Still Alive

#127

Post by Flenkno »

AlotOfReading wrote: post #37
nrp wrote: post #13

> Give AMD Engineers more compute and engineering resources to fix and improve the AMD ecosystem, they have very few internal gpu boxes relative to what Nvidia provides to their engineers. This is real. We’ve found ourselves having to give hardware to engineers at AMD because they’re unable to get allocation of it internally.

Sadly common at hardware companies. The most extreme case I've heard of is ASML, who supposedly doesn't keep any machines of their own. They test against "almost-ready" machines right before they go out the door to customers.

You are comparing a machine the size of a container with GPUs?

its nice to be aware of this but this is so fastly different from a critisism point of view that i don't think that matters.

Re: CUDA Moat Still Alive

#128

Post by Flenkno »

ryao wrote: post #54

> AMD is attempting to vertically integrate next year with their upcoming Pollara 400G NIC, which supports Ultra Ethernet, hopefully making AMD competitive with Nvidia. Infiniband is an industry standard. It is weird to see the industry invent yet another standard to do effectively the same thing just because Nvidia is using it. This “Nvidia does things this way so let’s do it differently” mentality is hurting AMD: *…

Nvidia has a unique problem, wants to move fast and has a shit load of money.

No need for Nvidia to go first to an industry standard and neither for AMD.

Personally would be great its getting backported but its so far away from an normal use case.

Re: CUDA Moat Still Alive

#129

Post by ryao »

mixblast wrote: post #120
ryao wrote: post #54

> AMD is attempting to vertically integrate next year with their upcoming Pollara 400G NIC, which supports Ultra Ethernet, hopefully making AMD competitive with Nvidia. Infiniband is an industry standard. It is weird to see the industry invent yet another standard to do effectively the same thing just because Nvidia is using it. This “Nvidia does things this way so let’s do it differently” mentality is hurting AMD: *…

Infiniband is a monopoly from NVidia (Mellanox). Everyone else would much rather use Ethernet which is the actual industry standard.

Others can build infiniband hardware according to the standard. There used to be at least two companies building infiniband hardware until Intel killed QLogic’s infiniband division in a misguided attempt to make its own monopoly. :/

Re: CUDA Moat Still Alive

#130

Post by ryao »

croes wrote: post #95
ryao wrote: post #54

> AMD is attempting to vertically integrate next year with their upcoming Pollara 400G NIC, which supports Ultra Ethernet, hopefully making AMD competitive with Nvidia. Infiniband is an industry standard. It is weird to see the industry invent yet another standard to do effectively the same thing just because Nvidia is using it. This “Nvidia does things this way so let’s do it differently” mentality is hurting AMD: *…

You hardly beat someone by copying him. They have way more experience in the field you try to catch up.

You don’t beat someone by doing everything worse either.
Post reply on HN