Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why isn't AMD's MI300X competitive?

newsletter.semianalysis.com

Post reply on HN
Print view View original

31–38 of 38 posts

Re: Why isn't AMD's MI300X competitive?

#31

Post by c0balt »

tripledry wrote: post #10
Havoc wrote: post #5

Mirrors the geohotz rants about AMD at the time, though as others point out this - 2024 - is ancient news in AI world and not quite sure what value it adds to the current discussions

Has this changed, If I want to go hands on with development using pytorch or whatever is used now, would you recommend an AMD card? Genuine question, I have not followed this topic closely for years :)

Short answer, no.

There are better learning resources and a better ecosystem available around Nvidia cards & software (cuda).

Long answer, it depends. It will add more challenges and require significantly more effort (even outside the GPU programming itself, debugging toolchain etc. is a somewhat separate skill). The smaller/less mature ecosystem also means you will have less examples to look at for references.

Re: Why isn't AMD's MI300X competitive?

#32

Post by c0balt »

HarHarVeryFunny wrote: post #24
dingdingdang wrote: post #11

Earlier quoted context omitted.

These days it's for sure the dev environment that is lacking, hardware is okay (potentially great?!), software abysmal. To run a local llm in a stable manner implies using Vulkan.. any attempt at ROCm is totally hamstrung by haphazard support of hardware alongside with an online presence poisoned by people primarily discussing work-arounds rather than work when it comes to AMD as a platform. Argh.

Is there any benefit of Vulcan vs ROCm on a card where ROCm is fully supported?

A vulkan computer shader is more portable, chances are also that the tooling for it will still be supported in a few years for your GPU (which isn't a given for ROCm especially when dealing with consumer cards)

Re: Why isn't AMD's MI300X competitive?

#33

Post by hgoel »

oneofthose wrote: post #19

> AMD’s software experience is riddled with bugs [...] AMD’s weaker-than-expected software Quality Assurance (QA) culture and its challenging out of the box experience. This has anecdotally been true since forever. Back in the day, OpenCL implementations were passing conformance test but performance was poor. They could not turn hardware capabilities into performance for compute users. Drivers were buggy. Documentati…

I think it's useful to consider that NVIDIA bet on CUDA early, they've supported it since 2006. AMD has to do a lot less work, but it's still going to take a while to get all of the software in a competitive state.

Though, on the other hand, I'm not very convinced AMD is even seriously trying, with how much of a mess ROCm has continued to be. GCN was an excellent GPU compute architecture, but they never seemed to manage to make much of that.

I had been willing to put up with the software support struggles too, but the way ROCm support for the Radeon VII and 5000 series had been handled really put me off.

Re: Why isn't AMD's MI300X competitive?

#35

Post by lhl »

tripledry wrote: post #10
Havoc wrote: post #5

Mirrors the geohotz rants about AMD at the time, though as others point out this - 2024 - is ancient news in AI world and not quite sure what value it adds to the current discussions

Has this changed, If I want to go hands on with development using pytorch or whatever is used now, would you recommend an AMD card? Genuine question, I have not followed this topic closely for years :)

RDNA is a whole different (and much poorer supported) animal than CDNA. As someone with extensive experience in both, if you're asking the question, then, no.

(If you're just looking to learn, use the free Kaggle/Google Cola T4s/TPUs to get started.)

Re: Why isn't AMD's MI300X competitive?

#36

Post by WhyNotHugo »

oneofthose wrote: post #19

> AMD’s software experience is riddled with bugs [...] AMD’s weaker-than-expected software Quality Assurance (QA) culture and its challenging out of the box experience. This has anecdotally been true since forever. Back in the day, OpenCL implementations were passing conformance test but performance was poor. They could not turn hardware capabilities into performance for compute users. Drivers were buggy. Documentati…

> AMD’s software experience is riddled with bugs [...] AMD’s weaker-than-expected software Quality Assurance (QA) culture and its challenging out of the box experience.

This was even true ~2005 in gaming circles. AMD drivers were buggy, so even when their cards were more performant at the same price point, folks opted for NVIDIA for reliability.

Re: Why isn't AMD's MI300X competitive?

#37

Post by WhyNotHugo »

threepts wrote: post #18

NVIDIA has such a big moat around their CUDA architecture such that I don't think AMD will ever be able to outcompete them in AI compute unless they somehow find 2-3 nobel prize level breakthroughs today.

I don’t think building a separate moat is ever going to work. They don’t just have to catch up and surpass in performance, but also justify customers moving out of one silo and into another vendor-lock in.

Re: Why isn't AMD's MI300X competitive?

#38

Post by lmm »

oneofthose wrote: post #19

> AMD’s software experience is riddled with bugs [...] AMD’s weaker-than-expected software Quality Assurance (QA) culture and its challenging out of the box experience. This has anecdotally been true since forever. Back in the day, OpenCL implementations were passing conformance test but performance was poor. They could not turn hardware capabilities into performance for compute users. Drivers were buggy. Documentati…

Cultural change in large organisations is hard to impossible. Like most hardware manufacturers, AMD culturally does not value software and never has. I've said before that it's not at all strange that AMD's drivers suck, the thing that's remarkable is that Nvidia has somehow managed to build a good software engineering culture that releases good drivers and libraries (at least relatively speaking).
Post reply on HN