Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CUDA Moat Still Alive

semianalysis.com

Post reply on HN
Print view View original

111–120 of 176 posts

Re: CUDA Moat Still Alive

#111

Post by sangnoir »

Twirrim wrote: post #84
sangnoir wrote: post #62

Earlier quoted context omitted.

> she deflects into "performance is what matters," which is the kind of denial that rhymes exactly with the problems they are having. It's not a deflection, but a straightforward description of AMDs current top-down market strategy of partnering with big players instead of doubling down to have a great OOBE for consumers & others who don't order GPUs by the pallet. It's an honest reflection if their current core comp…

Engineers at hyperscalers are struggling through all the bugs too. It's coming at notable opportunity cost for them, at a time when they also want an end to the monopoly. Do they buy AMD and wade through bug after bug, regression after regression, or do they shell out slightly more money for Nvidia GPUs and have it "just work". AMD has to get on top of their software quality issues if they're ever going to succeed in…

> Engineers at hyperscalers are struggling through all the bugs too

[citation needed]

Re: CUDA Moat Still Alive

#112

Post by pjmlp »

mountainriver wrote: post #26
nrp wrote: post #13

> Give AMD Engineers more compute and engineering resources to fix and improve the AMD ecosystem, they have very few internal gpu boxes relative to what Nvidia provides to their engineers. This is real. We’ve found ourselves having to give hardware to engineers at AMD because they’re unable to get allocation of it internally.

This seems so insane, is anyone actually doing the work to provide an alternative to CUDA? Maybe Google?

The problem is that everyone that tries, keeps missing the CUDA forest and focus only in a specific kind of tree.

Re: CUDA Moat Still Alive

#113

Post by pjmlp »

ryao wrote: post #48
roenxi wrote: post #19

Earlier quoted context omitted.

I expect everyone has been saying it for a while, the calls are just getting more strident and public as it becomes clear that AMD's failures are strategic rather than tactical. And as people try to build business on their half-hearted attempts. I still think it is a mistake to say that CUDA is a moat. IMO the problem here is that AMD still doesn't seem to think that GPGPU compute is a thing. They don't seem to under…

I wonder to what extent vulkan compute could be used for this. Of course, it is only an option on their RDNA GPUs since CDNA is not for graphics, even though that is the G in GPU.

Unless it provides the polyglot capabilities of CUDA, and related IDE and graphical debugging capabilities, not really.

Re: CUDA Moat Still Alive

#114

Post by pjmlp »

almostgotcaught wrote: post #42
dogma1138 wrote: post #20

Earlier quoted context omitted.

Anyone who looks at the mess that is ROCm and the design choices they made could easily see that. GPU support lagged behind for years, no support for APUs and no guaranteed forward compatibility were clear signs that as a whole they have no idea what they are doing when it comes to building and shipping a software ecosystem. To that you can add the long history of both AMD and ATI before they merged releasing dog shi…

> On the other hand you can take any CUDA binary even one that dates back to the original Tesla and run it on any modern NVIDIA GPU This particular difference stems the fact that NVIDIA has PTX and AMD does not have any such thing. Ie this kind of backwards compatibility will never be possible on AMD.

Backwards compatibility, and polyglot ecosystem, thanks to the amount of compiler toolchains that support PTX.

Re: CUDA Moat Still Alive

#115

Post by pjmlp »

cavisne wrote: post #92

I think it’s hardware not software. “Cuda moat” is a misnomer. The PTX spec is relatively short (600 page pdf). Triton directly writes PTX, skipping cuda. Flash attention was created by a non nvidia employee without access to any of the secret sauce within Cuda or its libraries. The hardware is just not as good, and no software can paper over its flaws.

NVidia also refactored their hardware design to follow C++ memory model, I think this still isn't the case of others.

Re: CUDA Moat Still Alive

#116

Post by david-gpu »

nemothekid wrote: post #81
sangnoir wrote: post #62

Earlier quoted context omitted.

> she deflects into "performance is what matters," which is the kind of denial that rhymes exactly with the problems they are having. It's not a deflection, but a straightforward description of AMDs current top-down market strategy of partnering with big players instead of doubling down to have a great OOBE for consumers & others who don't order GPUs by the pallet. It's an honest reflection if their current core comp…

> Hyperscalers are also more self-sufficient at software: they have entire teams working on PyTorch, Jax, and writing kernels. None of this matters because AMD drivers are broken. No one is asking AMD to write a PyTorch backend. The idea that AMD will have twice the silicon performance than nvidia to make up the performance loss for bad software is a pipedream.

> None of this matters because AMD drivers are broken

How do you know that the problems arise from broken drivers rather than broken hardware? Real world GPU drivers are full of workarounds for hardware bugs.

Re: CUDA Moat Still Alive

#117

Post by david-gpu »

anon291 wrote: post #72
mountainriver wrote: post #26

Earlier quoted context omitted.

This seems so insane, is anyone actually doing the work to provide an alternative to CUDA? Maybe Google?

Honestly, probably NVIDIA itself, since they contribute significantly to many open-source projects (MLIR), and also make their SoTA GEMM/Conv implementations open-source and available for study (Cutlass).

*> also make their SoTA GEMM/Conv implementations open-source and available for study (Cutlass)"

Cutlass is a fine piece of engineering, but it is not quite as good as their closed source libraries in real world workloads. There is secret sauce that is not open sourced.

Re: CUDA Moat Still Alive

#118

Post by Comma2976 »

pjmlp wrote: post #110
sangnoir wrote: post #80

Earlier quoted context omitted.

What exactly are they in denial about? They are aware that software is not a strength of theirs, so they partner with those who are great at it. Would you say AMD is "shitting the bed" by not building it's own consoles too? You know AMD could build a kick-ass console since they are doing the heavy-lifting for the Playstation, and the XBox[1] , but AMD knows as much as anybody that they don't have the skills to wrangl…

It would go nowhere, games history is full of great hardware that died because they failed to have a profitable ecosystem. Even Steam Deck is only a success, because it depends on Windows ecosystem, and the moment Microsoft decides it is enough, lets see how long it holds.

Steam Decks run on Arch Linux

Re: CUDA Moat Still Alive

#119

Post by 0-_-0 »

dhruvdh wrote: post #68
Lerc wrote: post #66

Earlier quoted context omitted.

By overhead I'm talking about the things that have to be done supplementary to the algorithm. While there are complex state-of-the-art algorithms, those algorithms exist for everyone. The overhead is the bit that had to be done to make the algorithm work. For instance for sorting a list of strings the algorithm might be quick sort. The overhead would be in the efficiency of your string compare. For matmul I'm not sur…

Which algorithm you pick for what shape of matrices is different and not straightforward to figure out. AMD currently wants you to “tune” ops and likely search for the right algorithm for your shapes while Nvidia has accurate heuristics for picking the right algorithm.

Nvidia's heuristics are not accurate, and it's not possible to achieve peak performance without search.

Re: CUDA Moat Still Alive

#120

Post by mixblast »

ryao wrote: post #54

> AMD is attempting to vertically integrate next year with their upcoming Pollara 400G NIC, which supports Ultra Ethernet, hopefully making AMD competitive with Nvidia. Infiniband is an industry standard. It is weird to see the industry invent yet another standard to do effectively the same thing just because Nvidia is using it. This “Nvidia does things this way so let’s do it differently” mentality is hurting AMD: *…

Infiniband is a monopoly from NVidia (Mellanox). Everyone else would much rather use Ethernet which is the actual industry standard.
Post reply on HN