Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

phoronix.com

Post reply on HN
Print view View original

381–390 of 425 posts

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#381

Post by pjmlp »

Avamander wrote: post #199
smoldesu wrote: post #124

Earlier quoted context omitted.

Those people should have spoken up when their hardware manufacturers abandoned OpenCL. The industry set itself 5-10 years behind by ignoring open GPGPU compute drivers while Nvidia slowly built their empire. Just look at how long it's taken to re-impliment a fraction of the CUDA featureset on a small handful of hardware. CUDA shouldn't exist. We should have hardware manufacturers working together , using common APIs…

Is my impression wrong, that people understood the need for OCL only after CUDA had already cornered and strangled the market?

I attended a Webminar from Khronos were no one in the panel understood why the research community would want anything beyond C to program GPUs.

Meanwhile NVidia was adding C++, Fortran, PTX, supporting other programming language communities trying to target GPUS (Java, .NET, Haskell,..).

Making it as easy to debug GPUs as modern graphical debuggers for CPUs, building libraries,...

Intel, and AMD together with Khronos did this to themselves.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#382

Post by Cu3PO42 »

washadjeffmad wrote: post #342
Cu3PO42 wrote: post #270

Earlier quoted context omitted.

I have heard that DirectML was a somewhat easier story, but allegedly has worse performance (and obviously it's Windows only...). But I'm not entirely suprised that setup is somewhat easier on Windows, where bundling everything is an accepted approach. With AMD's official 15GB(!) Docker image, I was now able to get the A1111 UI running. With SD 1.5 and 30 sample iterations, generating an image takes under 2s. I'm sti…

That has to include the model(s), no? Also, nothing is easier on Windows. It's a wonder that anything works there, except for the power of recalcitrance. Not dogging Windows users, but once your brain heals, it just can't go back.

It actually doesn't include the models! The image is Ubuntu with ROCm and a number of ML libraries, such as Torch, preinstalled.

> Also, nothing is easier on Windows.

As much as I, too, dislike Windows, I still have to disagree. I have encountered (proprietary) software which was much easier to get working on Windows. For example, Cisco AnyConnect with SmartCard authentication has been a nightmare for me on Linux.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#383

Post by pjmlp »

JonChesterfield wrote: post #287
bornfreddy wrote: post #281

Earlier quoted context omitted.

Yes. This is what makes Nvidia's toolchain, if not great, at least ok. As a developer I can actually use their GPUs. And what I developed locally I can yhen run on Nvidia hardware in the cloud and pay by usage. AMD doesn't seem to understand that affordable entry-level hardware with good software support is key.

Ah yes, so that one does seem to be a stumbling block. ROCm is not remotely convinced that running on gaming cards is a particularly useful thing. HN is really sure that being able to develop code on ~free cards that you've got lying around anyway is an important gateway to running on amdgpu. The sad thing is people can absolutely run ROCm on gaming cards if they build from source. Weirdly GPU programmers seem determ…

Strage world is expecting we are all Gentoo users, that like to build our software like making fire with stones and sticks.

Then people act surprised CUDA was won the hearts of the scientific developer community, that rather spend their time actually doing research work.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#384

Post by pjmlp »

ianlevesque wrote: post #237
mindcrime wrote: post #233

Earlier quoted context omitted.

Yep. This is very similar to the "catch-22" that IBM wound up in with OS/2 and the Windows API. On the one hand, by supporting Windows software on OS/2, they gave OS/2 customers access to a ready base of available, popular software. But in doing so, they also reduced the incentive for ISV's to produce OS/2 native software that could take advantage of unique features of OS/2. It's a classic "between a rock and a hard…

Thinking about the highly adjacent graphics APIs history, did anyone really 'win' the Direct3D, OpenGL, Metal, Vulkan war? Are we benefiting from the fragmentation? If the players in the space have naturally coalesced around one over the last decade, can we skip the thrashing and just go with it this time?

You missed the Nintendo and Sony APIs as well.

FOSS folks make this a bigger issue than it really is, game studios make a pluggable API on their engine and call it a day, move on into everything else that matters in actually delivering a game.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#385

Post by pests »

squigz wrote: post #380
squigz wrote: post #379

Earlier quoted context omitted.

Can you provide some context about this?

Found some... Seems crazy to me; this community has never felt transphobic to me... https://news.ycombinator.com/item?id=36226845

That is unhinged. I am at a loss after reading that thread. I always thought HN was a little over represented in the queer and furry communities.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#386

Post by Cu3PO42 »

doctorpangloss wrote: post #359
Cu3PO42 wrote: post #335

Earlier quoted context omitted.

Hi! I really appreciate you taking the time to reply. I have since gotten Invoke to run and was already able to get some results I'm really quite happy with, so thank you for your time and commitment working on Invoke! I understand that ROCm is still challenging, but it seems my problems were less related to ROCm or Invoke itself and more to Python dependency management. It really boiled down to getting the correct (…

> Installing Invoke from PyPi... To me, your pyproject.toml looks perfectly sane, so I wasn't sure how to go about fixing the problem. You can't install the PyTorch that's best for the currently running platform using a pyproject.toml with a setuptools backend, for starters. Invoke would have to author a setup.py that deals with all the issues, in a way that is compatible with build isolation. > The majority of my st…

> You can't install the PyTorch that's best for the currently running platform using a pyproject.toml with a setuptools backend, for starters.

I see. I do know Python, but my knowledge of setuptools, pip, poetry and whatever else have you. To get my working setup, I specified an --index-url for my Torch installation. Does that not work while using their current setup?

> Why? Given the state of the ecosystem, what guarantee is there really that the documentation for Docker Desktop with AMD ROCm device binding is going to actually work for your device?

Well, they did work for me. Though I think only passing /dev/{dri,kfd} and setting seccomp=unconfined was sufficient. So for my particular case, getting a working image was the only missing step.

From a more general POV: it might not make sense to invest in a ROCm OCI image from a short-term business perspective, but in the long term and based purely on principal, I do think the ecosystem should strive to be less reliant on CUDA and only CUDA.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#387

Post by Cu3PO42 »

sophrocyne wrote: post #354
Cu3PO42 wrote: post #335

Earlier quoted context omitted.

Hi! I really appreciate you taking the time to reply. I have since gotten Invoke to run and was already able to get some results I'm really quite happy with, so thank you for your time and commitment working on Invoke! I understand that ROCm is still challenging, but it seems my problems were less related to ROCm or Invoke itself and more to Python dependency management. It really boiled down to getting the correct (…

You bet - Thanks for the feedback. Glad you're enjoying Invoke! We do have Docker packages hosted on GH, but I'll be the first to admit that we haven't prioritized ROCm. Contributors who have AMDs are a scant few, but maybe we'll find some help in wrangling that problem now that we know there's an avenue to do so.

I hate maintaining my own build instructions as much as the next guy, so I'll try to get your Dockerfile working for me and then send a PR.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#388

Post by hnfong »

HarHarVeryFunny wrote: post #357
tgsovlerkhgsel wrote: post #313

Earlier quoted context omitted.

How is this not priority #1 for them, with NVIDIA stock shooting to the moon because everyone does machine learning using CUDA-centric tools? If AMD could get 90% of the CUDA ML stuff to seamlessly run on AMD hardware, and could provide hardware at a competitive cost-per-performance (which I assume they probably could since NVIDIA must have an insane profit margin on their GPUs), wouldn't that be the opportunity to e…

IMO the trouble is that CUDA is too low level to allow emulation without a major loss of performance, and even if there was a choice of CUDA-compatible vendors, people are ultimately going to vote with their wallets. It's not enough to be compatible - you need to be compatible while providing the same or better performance (else why not just use NVIDIA). A better level to target compatibility would be at the framewor…

> However, these frameworks are forever changing and playing continual catch-up there still wouldn't be a great place to be, especially without a large staff dedicated to the effort (writing hand-optimized kernels), which AMD don't seem to be able/willing to muster.

The situation in reality is quite actually quite bad.

Given that I have a M2 Max and no nVidia cards, I've tried enough PyTorch-based ML libraries that at some point, I basically expect them to flat out show an error saying CUDA 10.x+ is required once the dependencies are installed (eg. one of them being the bitsandbytes library -- in fairness, there's apparently some effort trying to port the code to other platforms as well).

As of today, the whole field is moving too fast that it's simply not worth it for a solo dev or even a small team to even attempt getting a non-CUDA stack up and running, especially with the other major GPU vendors not (able to?) hiring people to port the hand-optimized CUDA kernels.

Hopefully the situation will change after these couple years of frenzy, but in the time being I don't see any viable way to avoid using a CUDA stack if one is serious with getting ML stuff done.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#389

Post by PeterisP »

codedokode wrote: post #306

As I understand, Vulkan allows to run custom code on GPU, including the code to multiply matrices. Can one simply use Vulkan and ignore CUDA, PyTorch and ROCm?

You probably can, but why would you? The main (only?) reason to ignore the CUDA-based stack is so that you could save a bit of money by using some other hardware instead of nVidia. So the amount of engineering labor/costs you should be willing to accept is directly tied to how much hardware you intend to buy or rent and what % discount, if any, the alternative hardware enables compared to nVidia.

So if you'd want to ignore CUDA+PyTorch and reimplement all of what you need on top of Vulkan.... well, that becomes worthy of discussion only if you expect to spend a lot on hardware, if you really consider that savings on hardware can recoup many engineer-years of costs - otherwise it's more effective to just go with the flow.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#390

Post by Certhas »

kkielhofner wrote: post #308
Certhas wrote: post #117

Earlier quoted context omitted.

The most recent "card" is their MI300 line. It's annoying as hell to you and me that they are not catering to the market of people who want to run stuff on their gaming cards. But it's not clear it's bad strategy to focus on executing in the high-end first. They have been very successful landing MI300s in the HPC space... Edit: I just looked it up: 25% of the GPU Compute in the current Top500 Supercomputers is AMD ht…

Indeed, but this is extremely short-sighted. You don't win an overall market by focusing on several hundred million dollar bespoke HPC builds where the platform (frankly) doesn't matter at all. I'm working on a project on an AMD platform on the list (won't say - for now) and needless to say you build whatever you have to what's there, regardless of what it takes and the operators/owners and vendor support teams pour…

Those people didn't build the CUDA ecosystem. nVidia, Google and Facebook did. I think your hypothesis is pretty self-serving.

nVidia is dominant now. The question is, what's your wedge.

Post reply on HN