Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why isn't AMD's MI300X competitive?

newsletter.semianalysis.com

Post reply on HN
Print view View original

11–20 of 38 posts

Re: Why isn't AMD's MI300X competitive?

#11

Post by dingdingdang »

pstuart wrote: post #2

If AMD's betting the company on their AI compute, they had best follow the advice in the article because the only way to compete with NVIDIA is to meet/exceed not just the performance but also the DevX.

These days it's for sure the dev environment that is lacking, hardware is okay (potentially great?!), software abysmal. To run a local llm in a stable manner implies using Vulkan.. any attempt at ROCm is totally hamstrung by haphazard support of hardware alongside with an online presence poisoned by people primarily discussing work-arounds rather than work when it comes to AMD as a platform. Argh.

Re: Why isn't AMD's MI300X competitive?

#12

Post by KeplerBoy »

pstuart wrote: post #2

If AMD's betting the company on their AI compute, they had best follow the advice in the article because the only way to compete with NVIDIA is to meet/exceed not just the performance but also the DevX.

You can't have good performance without good DevX. There's a reason why we get a new python dsl for nvidia GPUs every week.

Re: Why isn't AMD's MI300X competitive?

#16

Post by andy_ppp »

joelthelion wrote: post #9
andy_ppp wrote: post #7

Please just get everything in PyTorch to work, and work well (and across all graphics cards too). This is the starting point and it doesn't matter how you do it. But the fact you cannot even do some very basic stuff on AMD is going to mean you are left unused by researchers, so getting further up the stack is going to be almost impossible.

The problem is "just". "Just" getting pytorch to work and to work well is a huge undertaking.

Just, in this case means “at minimum” or “first and foremost, no excuses”. I obviously understand this is a huge undertaking. Nobody said attempting to be competitive with NVIDIA in AI would be a walk in the park.

Re: Why isn't AMD's MI300X competitive?

#17

Post by Havoc »

tripledry wrote: post #10
Havoc wrote: post #5

Mirrors the geohotz rants about AMD at the time, though as others point out this - 2024 - is ancient news in AI world and not quite sure what value it adds to the current discussions

Has this changed, If I want to go hands on with development using pytorch or whatever is used now, would you recommend an AMD card? Genuine question, I have not followed this topic closely for years :)

Still rocking a 3090 so can't speak from experience but general vibe around simple at home inference seems like it has improved (esp since both vulkan and rocm are now viable paths on newer cards).

>development using pytorch

Would probably still play it nvidia safe for more adventurous stuff than token generation even if it has improved

Re: Why isn't AMD's MI300X competitive?

#19

Post by oneofthose »

> AMD’s software experience is riddled with bugs [...] AMD’s weaker-than-expected software Quality Assurance (QA) culture and its challenging out of the box experience.

This has anecdotally been true since forever. Back in the day, OpenCL implementations were passing conformance test but performance was poor. They could not turn hardware capabilities into performance for compute users. Drivers were buggy. Documentation was poor compared to NVidia's docs and forum. Offerings were inconsistent (look up Sycl from Codeplay) and ownership of what it is like to develop for AMD was unclear. The notion that it might not have improved or is only now improving is puzzling. It can't be for the lack of recognizing the problem. Intuitively it does not seem so difficult. I'm curious what the reasons are.

Re: Why isn't AMD's MI300X competitive?

#20

Post by blitzar »

joelthelion wrote: post #9
andy_ppp wrote: post #7

Please just get everything in PyTorch to work, and work well (and across all graphics cards too). This is the starting point and it doesn't matter how you do it. But the fact you cannot even do some very basic stuff on AMD is going to mean you are left unused by researchers, so getting further up the stack is going to be almost impossible.

The problem is "just". "Just" getting pytorch to work and to work well is a huge undertaking.

for a trillion dollars, they should be able to figure it out.
Post reply on HN