Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

phoronix.com

Post reply on HN
Print view View original

351–360 of 425 posts

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#351

Post by tormeh »

llm_trw wrote: post #331
tgsovlerkhgsel wrote: post #313

Earlier quoted context omitted.

How is this not priority #1 for them, with NVIDIA stock shooting to the moon because everyone does machine learning using CUDA-centric tools? If AMD could get 90% of the CUDA ML stuff to seamlessly run on AMD hardware, and could provide hardware at a competitive cost-per-performance (which I assume they probably could since NVIDIA must have an insane profit margin on their GPUs), wouldn't that be the opportunity to e…

Never underestimate AMD's ability to fail. Ryzen was a surprise to everyone not because it was good, but because they didn't fuck it up within two generations. AMD cards have more raw compute than nvidia, they are better than nvidia, yet the software is so bad that I gave up on using it and switched to nvidia. Two weeks of debugging driver errors vs 30 minutes of automated updates.

It's rather shocking that with RADV, Valve (mostly) has written a better RDNA2 driver than AMD has managed for their own cards. Besides the embarrassment, AMD is leaving tons of performance and therefore market share on the table. You have to wonder wtf is going on over at AMD.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#352

Post by exikyut »

Rafuino wrote: post #327
pheatherlite wrote: post #316

Earlier quoted context omitted.

The only reason our lab bought 20k worth of Nvidia gpu cards rather than amd was the cuda industry standard (might as wellbe). It's kind of mind boggling how much business amd must be losing over this.

So, your lab bought ~1 GPU?

Hey, I should go play with those workstation/server configurators now they'll have been updated to supply A100Xs and such...

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#353

Post by dralley »

tormeh wrote: post #351
llm_trw wrote: post #331

Earlier quoted context omitted.

Never underestimate AMD's ability to fail. Ryzen was a surprise to everyone not because it was good, but because they didn't fuck it up within two generations. AMD cards have more raw compute than nvidia, they are better than nvidia, yet the software is so bad that I gave up on using it and switched to nvidia. Two weeks of debugging driver errors vs 30 minutes of automated updates.

It's rather shocking that with RADV, Valve (mostly) has written a better RDNA2 driver than AMD has managed for their own cards. Besides the embarrassment, AMD is leaving tons of performance and therefore market share on the table. You have to wonder wtf is going on over at AMD.

RADV was started by David Arlie of Red Hat, although Valve been dedicating some very significant resources over the past few years.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#354

Post by sophrocyne »

Cu3PO42 wrote: post #335
sophrocyne wrote: post #307

Earlier quoted context omitted.

Hey there - I'm a maintainer (and CEO) of Invoke. It's something we're monitoring as well. ROCm has been challenging to work with - we're actively talking to AMD to keep apprised of ways we can mitigate some of the more troublesome experiences that users have with getting Invoke running on AMD (and hoping to expand official support to Windows AMD) The problem is that a lot of the solutions proposed involve significan…

Hi! I really appreciate you taking the time to reply. I have since gotten Invoke to run and was already able to get some results I'm really quite happy with, so thank you for your time and commitment working on Invoke! I understand that ROCm is still challenging, but it seems my problems were less related to ROCm or Invoke itself and more to Python dependency management. It really boiled down to getting the correct (…

You bet - Thanks for the feedback. Glad you're enjoying Invoke!

We do have Docker packages hosted on GH, but I'll be the first to admit that we haven't prioritized ROCm. Contributors who have AMDs are a scant few, but maybe we'll find some help in wrangling that problem now that we know there's an avenue to do so.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#355

Post by dralley »

bigdict wrote: post #290
smcl wrote: post #261

Earlier quoted context omitted.

I know that Lina doesn't like a lot of the attention HN sends her way so it may be better if you don't link her socials here.

Sounds ridiculous, why have a public presence on a social network then?

IIRC, it's not that she doesn't like the attention to her work, it's that she doesn't like how quickly conversations get derailed by things that have nothing to do with the work.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#356

Post by yen223 »

RachelF wrote: post #329
Keyframe wrote: post #173

This event of release is however a result of AMD stopped funding it per "After two years of development and some deliberation, AMD decided that there is no business case for running CUDA applications on AMD GPUs. One of the terms of my contract with AMD was that if AMD did not find it fit for further development, I could release it. Which brings us to today." from https://github.com/vosen/ZLUDA?tab=readme-ov-file#faq…

Yeah, AMD look like idiots for doing this. Either they are very stupid, or open sourcing the library stops NVidia from suing them in a repeat of the Oracle/Google lawsuit over Java APIs? I'm not sure what the reason is?

I think AMD is focusing on the "inference" side of ML, which doesn't really require CUDA or similar, and is what they believe a much larger market.

Time will tell if that strategy is going to pan out. Ceding the ML "training" market entirely to Nvidia is certainly a bold move

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#357

Post by HarHarVeryFunny »

tgsovlerkhgsel wrote: post #313
Keyframe wrote: post #173

This event of release is however a result of AMD stopped funding it per "After two years of development and some deliberation, AMD decided that there is no business case for running CUDA applications on AMD GPUs. One of the terms of my contract with AMD was that if AMD did not find it fit for further development, I could release it. Which brings us to today." from https://github.com/vosen/ZLUDA?tab=readme-ov-file#faq…

How is this not priority #1 for them, with NVIDIA stock shooting to the moon because everyone does machine learning using CUDA-centric tools? If AMD could get 90% of the CUDA ML stuff to seamlessly run on AMD hardware, and could provide hardware at a competitive cost-per-performance (which I assume they probably could since NVIDIA must have an insane profit margin on their GPUs), wouldn't that be the opportunity to e…

IMO the trouble is that CUDA is too low level to allow emulation without a major loss of performance, and even if there was a choice of CUDA-compatible vendors, people are ultimately going to vote with their wallets. It's not enough to be compatible - you need to be compatible while providing the same or better performance (else why not just use NVIDIA).

A better level to target compatibility would be at the framework level such as PyTorch, where the building blocks of neural networks (convolution, multi-head attention, etc, etc) are high level and abstract enough to allow flexibility in mapping them onto AMD hardware without compromising performance.

However, these frameworks are forever changing and playing continual catch-up there still wouldn't be a great place to be, especially without a large staff dedicated to the effort (writing hand-optimized kernels), which AMD don't seem to be able/willing to muster.

So, finally, perhaps the strategically best place for AMD to invest would be in compilers and software tools to allow kernels to be written in a high level language. Becoming a first class Mojo target wouldn't be a bad place to start, assuming they are not already in partnership.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#358

Post by croutons »

weebull wrote: post #299
saboot wrote: post #105

Earlier quoted context omitted.

Yep, I develop several applications that use CUDA. I see AMD/Radeon powered computers for sale and want to buy one, but I am not going to risk not being able to run those applications or having to rewrite them. If they want me as a customer, and they have not created a viable alternative to CUDA, they need to pursue this.

Define "viable"?

A backend that runs PyTorch out of the box and is as easy to setup / use as nvidia stack.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#359

Post by doctorpangloss »

Cu3PO42 wrote: post #335
sophrocyne wrote: post #307

Earlier quoted context omitted.

Hey there - I'm a maintainer (and CEO) of Invoke. It's something we're monitoring as well. ROCm has been challenging to work with - we're actively talking to AMD to keep apprised of ways we can mitigate some of the more troublesome experiences that users have with getting Invoke running on AMD (and hoping to expand official support to Windows AMD) The problem is that a lot of the solutions proposed involve significan…

Hi! I really appreciate you taking the time to reply. I have since gotten Invoke to run and was already able to get some results I'm really quite happy with, so thank you for your time and commitment working on Invoke! I understand that ROCm is still challenging, but it seems my problems were less related to ROCm or Invoke itself and more to Python dependency management. It really boiled down to getting the correct (…

> Installing Invoke from PyPi... To me, your pyproject.toml looks perfectly sane, so I wasn't sure how to go about fixing the problem.

You can't install the PyTorch that's best for the currently running platform using a pyproject.toml with a setuptools backend, for starters. Invoke would have to author a setup.py that deals with all the issues, in a way that is compatible with build isolation.

> The majority of my struggle would have been solved by a recent working Docker image containing a working setup. (The one on Docker Hub is 9 months old.)

Why? Given the state of the ecosystem, what guarantee is there really that the documentation for Docker Desktop with AMD ROCm device binding is going to actually work for your device? (https://rocm.docs.amd.com/projects/MIVisionX/en/latest/docke...)

There is a lot of ad-hoc reinvention of tooling in this space.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#360

Post by coderenegade »

BizarroLand wrote: post #184
incrudible wrote: post #115

Earlier quoted context omitted.

Windows before NT was crap, so users had an incentive to upgrade. If there had existed a Windows 7 alternative that was near fully compatible and FOSS, I would wager Microsoft would have lost to it with Windows 8 and even 10. The only reason to update for most people was Microsoft dropping support. For CUDA, it is not just AMD who would need to catch up. Developers also are not necessarily going to target the latest…

Todays linux OS's would have competed incredibly strongly against Vista and probably would have gone blow for blow against 7. Proton, Wine, and all of the compatibility fixes and drive improvements that the community has made in the last 16 years has been amazing, and every day is another day where you can say that it has never been easier to switch away from Windows. However, Microsoft has definitely been drinking t…

I don't think it'll be a killer app so much as a confluence of different factors. For one thing, we now live in a world where docker is fast becoming as ubiquitous as git, and unlike git, requires a Linux VM to run on Windows. It's also a key technology for the replication and distribution of ML models, which again, are developed on Linux, trained on clusters running Linux, and deployed to servers running Linux. And this is all done in Python, a language native to Linux, which is now one of the most used languages on Earth.

We already see things like Google abandoning tensorflow support for Windows, because they don't have enough devs using Windows to easily maintain it.

And of course, we have a changing of the guard in terms of a generation of software developers who primarily worked on Windows, because that was the way to do it, starting to retire. Younger devs came up in the Google era where Linux is a first class citizen alongside MacOS.

I think these factors are going to change the face of technology in the coming 15 years, and that's likely to affect how businesses and consumers consume technology, even if they don't understand what's actually running under the hood.

Post reply on HN