Live data from Hacker News

AMD may get across the CUDA moat

hpcwire.com

121–130 of 312 posts

Re: AMD may get across the CUDA moat

#121

Earlier quoted context omitted.

Sigh. It's great that these container images exist to give people an easy on-ramp, but they definitely don't work for every use case (especially once you're in embedded where space matters and you might not be online to pull multi-gb updates from some registry). So it's important that vendors don't feel let off the hook to provide sane packaging just because there's an option to use a kitchen-sink container image the…

I know it's still different than what you're looking for, so you probably already know this, but many projects like this have the Dockerfile on github which shows exactly how they set up the image. For example: https://github.com/RadeonOpenCompute/ROCm-docker/blob/master... They also have some for Fedora. Looks like for this you need to install their repo: curl -sL https://repo.radeon.com/rocm/rocm.gpg.key | apt-key…

Oh yeah, I mean... having the source for the container build is kind of table stakes at this point. No one would accept a 10gb mystery meat blob as the basis of their production system. It's bad enough that we still accept binary-only drivers and proprietary libraries like TensorRT.

I think my issue is more just with the mindset that it's okay to have one narrow slice of supported versions of everything that are "known to work together" and those are what's in the container and anything outside of those and you're immediately pooched.

This is not hypothetical btw, I've run into real problems around it with libraries like gproto, where tensorflow's bazel build pulls in an exact version that's different from the default one in nixpkgs, and now you get symbol conflicts when something tries to link to the tensorflow c++ API while linking to another component already using the default gproto. I know these problems are solveable with symbol visibility control and whatever, but that stuff is far from universal and hard to get right, especially if the person setting up the build rules for the library doesn't themselves use it in that type of heterogeneous environment (like, everyone at Google just links the same global proto version from the monorepo so it doesn't matter).

Re: AMD may get across the CUDA moat

#122
post #114
post #57

Earlier quoted context omitted.

>if this unicorn were to show up A unicorn like that showed up a couple hours ago. Someone posted a guide for getting llama to run on a 7900xtx https://old.reddit.com/r/LocalLLaMA/comments/170tghx/guide_i... It's still slow and janky but this really isn't that far away. I don't buy that AMD can't make this happen if they actually tried. Go on fiverr, get them to compile a list of top 100 people in the DIY LLM space,…

llama.cpp, ExLlama, and MLC LLM have all had ROCm inferencing for months (here are a bunch of setup instructions I've written up, for Linux and Windows: https://llm-tracker.info/books/howto-guides/page/amd-gpus ) - but I don't think that's the problem (and wouldn't drive lots of volume or having downstream impact in any case). The bigger problem is on the training/research support. Eg, here's no official support for…

>an AMD employee is paying out of pocket for AMD GPUs

I hope he's at least getting an employee discount! I guess AMD is not a fan of the 20% concept either

Re: AMD may get across the CUDA moat

#124

Earlier quoted context omitted.

> especially once you're in embedded is this a real problem? exactly which embedded platform has a device that ROCm supports?

Robotic perception is the one relevant to me. You want to do object recognition on an industrial x86 or Jetson-type machine, without having to use Ubuntu or whatever the one "blessed" underlay system is (either natively or implicitly because you pulled a container based on it).

>industrial x86 or Jetson-type machine

that's not embedded dev. if you

1. use underpowered devices to perform sophisticated tasks

2. using code/tools that operate at extremely high levels of "abstraction"

don't be surprised when all the inherent complexity is tamed using just more layers of "abstraction". if that becomes a problem for your cost/power/space budget then reconsider choice 1 or choice 2.

Re: AMD may get across the CUDA moat

#125
post #42
post #26

CUDA is the only reason I have an Nvidia card, but if more projects start migrating to a more agnostic environment, I'll be really grateful. Running Nvidia in Linux isn't as much fun. Fedora and Debian can be incredibly reliable systems, but when you add an Nvidia card, I feel like I am back in Windows Vista with kernel crashes from time to time.

Yeah, nvidia linux support is meh, but still much better than amd.

In the closed source days of fglrx or whatever it's called I'd agree. Since they went open source, hard disagree. AMD graphics work in Linux about as well as Intel always has.

Re: AMD may get across the CUDA moat

#126
post #42
post #26

CUDA is the only reason I have an Nvidia card, but if more projects start migrating to a more agnostic environment, I'll be really grateful. Running Nvidia in Linux isn't as much fun. Fedora and Debian can be incredibly reliable systems, but when you add an Nvidia card, I feel like I am back in Windows Vista with kernel crashes from time to time.

Yeah, nvidia linux support is meh, but still much better than amd.

>> Yeah, nvidia linux support is meh, but still much better than amd.

Can not confirm. I used nvidia for years when it was the only option. Then used the nouveau driver on a well supported card because it worked well and eliminated hassle. Now I'm on AMD APU and it just works out of the box. YMMV of course. We do get reports of issues with AMD on specific driver versions, but I can't reproduce.

Re: AMD may get across the CUDA moat

#127

Earlier quoted context omitted.

Robotic perception is the one relevant to me. You want to do object recognition on an industrial x86 or Jetson-type machine, without having to use Ubuntu or whatever the one "blessed" underlay system is (either natively or implicitly because you pulled a container based on it).

>industrial x86 or Jetson-type machine that's not embedded dev. if you 1. use underpowered devices to perform sophisticated tasks 2. using code/tools that operate at extremely high levels of "abstraction" don't be surprised when all the inherent complexity is tamed using just more layers of "abstraction". if that becomes a problem for your cost/power/space budget then reconsider choice 1 or choice 2.

Not sure this is worth an argument over semantics, but modern "embedded" development is a lot bigger than just microcontrollers and wearables. IMO as soon as you're deploying a computer into any kind of "appliance", or you're offline for periods of time, or you're running on batteries or your primary network connection is wireless... then yeah, you're starting to hit the requirements associated with embedded and need to seek established solutions for them, including using distros which account for those requirements.

Re: AMD may get across the CUDA moat

#128

Earlier quoted context omitted.

> Best of all is that I simply set the device to `torch.device('cuda')` rather than openCL, which does wonders for compatibility Man oh man where did we go wrong that cuda is the more compatible option over OpenCL?

It must be a misnomer on PyTorch's side. Clearly it's neither CUDA nor OpenCL. AMD should just get it's shit together. This is ridiculous. Not the name, but the fact that you can only do FP64 on a GPU. Everybody is moving to FP16 and AMD is stuck on doubles?

I believe the fp64 limitation came from the laptop-grade GPU I had rather than inherent to AMD or ROCm.

The API level I could target was at least two or three versions behind the latest they have to offer.

Re: AMD may get across the CUDA moat

#129
post #52

I was able to use ROCm recently with Pytorch and after pulling some hair it worked quite well. The Radeon GPU I had on hand was a bit old and underpowered (RDNA2) and it only supported matmul on fp64, but for the job I needed done I saw a 200x increase in it/s over CPU despite the need to cast everywhere, and that made me super happy. Best of all is that I simply set the device to `torch.device('cuda')` rather than o…

Have you gotten it to work with Whisper by any chance?

Re: AMD may get across the CUDA moat

#130
post #26

CUDA is the only reason I have an Nvidia card, but if more projects start migrating to a more agnostic environment, I'll be really grateful. Running Nvidia in Linux isn't as much fun. Fedora and Debian can be incredibly reliable systems, but when you add an Nvidia card, I feel like I am back in Windows Vista with kernel crashes from time to time.

My Arch system would occasionally boot to a black screen. When this happened, no amount of tinkering could get it back. I had to reinstall the whole OS.

Turns out it was a conflict between nvidia drivers and my (10 year old) Intel integrated GPU. But once I switched to an AMD card, everything works flawlessly.

Ubuntu based systems barely worked at all. Incredibly unstable and would occasionally corrupt the output and barf colors and fragments of the desktop all over my screens.

AMD on arch has been an absolute delight. It just. Works. It's more stable than nvidia on windows.

For a lot of reasons-- but mainly Linux drivers-- I've totally sworn off nvidia cards. AMD just works better for me.

Post reply on HN