Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD may get across the CUDA moat

hpcwire.com

Post reply on HN
Print view View original

231–240 of 312 posts

Re: AMD may get across the CUDA moat

#231

Post by [deleted] »

josephg wrote: post #214
mgaunard wrote: post #193

Earlier quoted context omitted.

I don't know what world you live in, but this is a problem for any software development. You need to ensure that there is only one version of any library used globally throughout the code and that the set of versions is compatible with each other, and preferably you also want everything to be built against the same toolchain with the same flags. That usually means onboarding third-party libraries into your own build…

It’s not a universal problem. A lot of modern languages allow multiple versions of a library to be pulled in to the same code base, through different dependency paths. (Eg nodejs, rust). It’s not a perfect answer by any means, but it’s nice not needing to worry about some package pulling in an inconvenient version of one of its dependencies. Also, just to name it, it’s ridiculous that a specific graphics card manages…

[deleted]

Re: AMD may get across the CUDA moat

#232

Post by iopq »

roenxi wrote: post #143
Eisenstein wrote: post #87

Earlier quoted context omitted.

People don't usually get employed to make things with no demand, and people who work for companies with a budget line don't really care how much the nVidia tax is. You can thank hobbyists for creating a lot of demand for compatability with other cards.

There is huge demand for AMD cards that can efficiently multiply matrices together. The issue is that while there are currently isolated cases where people can make them do that, it doesn't seem to be possible at the scale that it needs to happen at. AMD are being dragged along by the market. Willingly, they aren't fighting it, but their focus has been on other areas.

Look at the earnings call:

https://www.fool.com/earnings/call-transcripts/2023/08/01/ad...

it's literally ALL AI, server, enterprise talk - AI is mentioned 64 times

AMD literally doesn't care about gaming anymore, server is their primary focus

Re: AMD may get across the CUDA moat

#233

Post by anthk »

mgaunard wrote: post #193
mikepurvis wrote: post #121

Earlier quoted context omitted.

Oh yeah, I mean... having the source for the container build is kind of table stakes at this point. No one would accept a 10gb mystery meat blob as the basis of their production system. It's bad enough that we still accept binary-only drivers and proprietary libraries like TensorRT. I think my issue is more just with the mindset that it's okay to have one narrow slice of supported versions of everything that are "kno…

I don't know what world you live in, but this is a problem for any software development. You need to ensure that there is only one version of any library used globally throughout the code and that the set of versions is compatible with each other, and preferably you also want everything to be built against the same toolchain with the same flags. That usually means onboarding third-party libraries into your own build…

That's trivial with Guix.

Re: AMD may get across the CUDA moat

#234

Post by Flameancer »

viewtransform wrote: post #151
roenxi wrote: post #143

Earlier quoted context omitted.

There is huge demand for AMD cards that can efficiently multiply matrices together. The issue is that while there are currently isolated cases where people can make them do that, it doesn't seem to be possible at the scale that it needs to happen at. AMD are being dragged along by the market. Willingly, they aren't fighting it, but their focus has been on other areas.

They've shifted a large pool of experienced engineers from legacy software projects to AI and moved the team under a veteran Xilinx AI director. Fingers crossed we should see significant changes in 2024.

As a new owner of a 7800XT I’m excited.

Re: AMD may get across the CUDA moat

#235

Post by anthk »

anuraaga wrote: post #219
mgaunard wrote: post #193

Earlier quoted context omitted.

I don't know what world you live in, but this is a problem for any software development. You need to ensure that there is only one version of any library used globally throughout the code and that the set of versions is compatible with each other, and preferably you also want everything to be built against the same toolchain with the same flags. That usually means onboarding third-party libraries into your own build…

I'd say with semver becoming far better known, this is not a problem for "any" software development. The developer gets the choice to pick libraries that are stable, often also influencing language choice. Mistakes happen, Guava broke the Java ecosystem for about two years, but it's never something that is accepted as just a fact of software development, it is a mistake. Wanting to hold Python+C ecosystem more accoun…

This would be the work for Guix. Much better than docker, and exportable to a lot of formats. Or just build a vm from the CLI, an ad-hoc environment, a Docker export or a direct rootfs to deploy and run in any compatible machine.

Re: AMD may get across the CUDA moat

#236

Post by errnoh »

kkielhofner wrote: post #190
RockRobotRock wrote: post #129

Earlier quoted context omitted.

Have you gotten it to work with Whisper by any chance?

Whisper is actually a great example of why Nvidia has such a stronghold on ML/AI and why it’s so difficult to compete. There’s getting something to “work”, which is often enough of a challenge with ROCm. Then there’s getting it to work well (next challenge). Then there’s getting it to work as well as Nvidia/CUDA. With Whisper, as one example, you should be running it with ctranslate2[0]. Of all the platforms on their…

While I agree that it's much more effort to get things working on AMD cards than it is with Nvidia, I was a bit surprised to see this comment mention Whisper being an example of "5-10x as performant".

https://www.tomshardware.com/news/whisper-audio-transcriptio... is a good example of Nvidia having no excuses being double the price when it comes to Whisper inference, with 7900XTX being directly comparable with 4080, albeit with higher power draw. To be fair it's not using ROCm but Direct3D 11, but for performance/price arguments sake that detail is not relevant.

EDIT: Also using CTranslate2 as an example is not great as it's actually a good showcase why ROCm is so far behind CUDA: It's all about adapting the tech and getting the popular libraries to support it. Things usually get implemented in CUDA first and then would need additional effort to add ROCm support that projects with low amount of (possibly hobbyist) maintainers might not have available. There's even an issue in CTranslate2 where they clearly state no-one is working to get ROCm supported in the library. ( https://github.com/OpenNMT/CTranslate2/issues/1072#issuecomm... )

Re: AMD may get across the CUDA moat

#237

Post by anthk »

gymbeaux wrote: post #155
javchz wrote: post #26

CUDA is the only reason I have an Nvidia card, but if more projects start migrating to a more agnostic environment, I'll be really grateful. Running Nvidia in Linux isn't as much fun. Fedora and Debian can be incredibly reliable systems, but when you add an Nvidia card, I feel like I am back in Windows Vista with kernel crashes from time to time.

I often have issues booting to the installer or first boot after install with an NVidia GPU. Pop_OS, Fedora and OpenSUSE work out of the box. Those are all Wayland I believe. Debian/Ubuntu distros are a bad time. I think they’re still X11. It’s ironic because X11 is supposed to be the more stable window manager.

X11 is not a window manager.

Re: AMD may get across the CUDA moat

#238

Post by Flameancer »

hot_gril wrote: post #80
fluxem wrote: post #65

I call it the 90% problem. If AMD works for 90% of my projects, I would still buy NVIDIA, which works for 100%, even though I’m paying a premium

I'm lazy, so it's 99% for me. I don't even mess with AMD CPUs; I know they're not exactly the same instruction set as Intel, and more importantly they work with a different (and less mainstream) set of mobos, so I don't want em. If AMD manages to pull more customers their way, that's great, it just means lower Intel premium for me.

What mainstream board company is intel only? Maybe a decade ago on AM3(+) but on AM5/AM5 I haven’t seen a main board partner not offer the same board SKU that works with Intel and AMD.

Re: AMD may get across the CUDA moat

#239

Post by Flameancer »

tomsmeding wrote: post #58
ginko wrote: post #23

Earlier quoted context omitted.

When I try to install rocm-ml-sdk on Arch linux it'll tell me the total installed size would be about 18GB. What can possibly explain this much bloat for what should essentially be a library on top of a graphics driver as well as some tools (compiler, profiler etc.)? A couple hundred MB I could understand if they come with graphical apps and demos, but not this..

A regular TensorFlow installation, just the Python library, is an 184 MB wheel that unpacks to about 1.2 GB of stuff. I have no clue what mess goes in there, but it's a lot. Still, if you're right that this package seems to take 18 GB disk size, something weird is going on.

He’s not wrong. I did a new arch install to try and get a 7800XT working with ROCm and PyTorch and was concussed on how I ran out of space but saw that ROCm was 18GB.

Re: AMD may get across the CUDA moat

#240

Post by JonChesterfield »

mikepurvis wrote: post #121
xahrepap wrote: post #105

Earlier quoted context omitted.

I know it's still different than what you're looking for, so you probably already know this, but many projects like this have the Dockerfile on github which shows exactly how they set up the image. For example: https://github.com/RadeonOpenCompute/ROCm-docker/blob/master... They also have some for Fedora. Looks like for this you need to install their repo: curl -sL https://repo.radeon.com/rocm/rocm.gpg.key | apt-key…

Oh yeah, I mean... having the source for the container build is kind of table stakes at this point. No one would accept a 10gb mystery meat blob as the basis of their production system. It's bad enough that we still accept binary-only drivers and proprietary libraries like TensorRT. I think my issue is more just with the mindset that it's okay to have one narrow slice of supported versions of everything that are "kno…

> No one would accept a 10gb mystery meat blob as the basis of their production system

Well, except for cuda. Which is a massive pile of proprietary software that people are using in production anyway.

Post reply on HN