Live data from Hacker News

AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

phoronix.com

201–210 of 425 posts

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#201
post #177

Earlier quoted context omitted.

PTX is just an ISA too. Programming languages annd ISA representations are effectively fungible, that’s the lesson of Microsoft CLR/Intermediate Language and Java too. A “machine” is a hardware and a language.

PTX is not a hardware ISA though, it's still software and can change more rapidly.

Not without breaking the support contract? If you change PTX format then CUDA 1.0 machines can no longer it and it's no longer PTX.

Again, you are missing the point. Java is both a language (java source) and a machine (the JVM). The latter is a hardware ISA - there are processors that implement Java bytecode as their ISA format. Yet most people who are running Java are not doing so on java-machine hardware, yet they are using the java ISA in the process.

https://en.wikipedia.org/wiki/Java_processor

https://en.wikipedia.org/wiki/Bytecode#Execution

any bytecode is an ISA, the bytecode spec defines the machine and you can physically build such a machine that executes bytecode directly. Or you can translate via an intermediate layer, like how Transmeta Crusoe processors executed x86 as bytecode on a VLIW processor (and how most modern x86 processors actually use RISC micro-ops inside).

these are completely fungible concepts. They are not quite the same thing but bytecode is clearly an ISA in itself. Any given processor can choose to use a particular bytecode as either an ISA or translate it to its native representation, and this includes both PTX, Java, and x86 (among all other bytecodes). And you can do the same for any other ISA (x86 as bytecode representation, etc).

furthermore, what most people think of as "ISAs" aren't necessarily so. For example RDNA2 is an ISA family - different processors have different capabilities (for example 5500XT has mesh shader support while 5700XT does not) and the APUs use a still different ISA internally etc. GFX1101 is not the same ISA as GFX1103 and so on. These are properly implementations not ISAs, or if you consider it to be an ISA then there is also a meta-ISA encompassing larger groups (which also applies to x86's numerous variations). But people casually throw it all into the "ISA" bucket and it leads to this imprecision.

like many things in computing, it's all a matter of perspective/position. where is the boundary between "CMT core within a 2-thread module that shares a front-end" and "SMT thread within a core with an ALU pinned to one particular thread"? It's a matter of perspective. Where is the boundary of "software" vs "hardware" when virtually every "software" implementation uses fixed-function accelerator units and every fixed-function accelerator unit is running a control program that defines a flow of execution and has schedulers/scoreboards multiplexing the execution unit across arbitrary data flows? It's a matter of perspective.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#202

Earlier quoted context omitted.

IBM was also incompetent and the os/2 team in Boca was had some exceptional engineers but was packed witg mostly mediocre-to-bad ones, which is why so many things in OS/2 were bad and why IBM got upset for Microsoft contributing negative work to the project because their lines of code contribution was negative (they were rewriting a lot of inefficient bloated IBM code). A lot went wrong with os/2. For CUDA, I think a…

You would think that by now AMD realizes that poor software is what left them behind in the dust, and would have changed that mindset.

Most businesses understand the pain points of their suppliers very well, as they feel that pain and gave themselves organized around it.

They have a hard time to understand the pain points of their consumers, as they don't feel that pain, look trough their own organisation-coloured glases, and can't see the real pain points from the whiney-customer ones.

AMD probably thinks software ecosystems are the easy part, ready to take it on whenever they feel like it and throw a token amount at it. They've built a great engine, see the carossery as beneath them, and don't understand why the lazy customer wants them to build the rest of the car too.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#203
post #63

Earlier quoted context omitted.

The problem with effectively supporting CUDA is that encourages CUDA adoption all the more strongly. Meanwhile, AMD will always be playing catch-up, forever having to patch issues, work around Nvidia/AMD differences, and accept the performance penalty that comes from having code optimised for another vendor's hardware. AMD needs to encourage developers to use their own ecosystem or an open standard.

There are some great replies to my comment - my original comment was too reductive. However, I still think that entrenching CUDA as the de-facto language for heterogeneous computing is a mistake. We need an open ecosystem for AI and HPC, where vendors compete on producing the best hardware.

The problem with open standards is that someone has to write them.

And that someone usually isn't a manufacturer, lest the committee be accused of bias.

Consequently, you get (a) outdated features that SotA has already moved beyond, (b) designed in a way that doesn't correspond to actual practice, and (c) that are overly generalized.

There are some notable exceptions (e.g. IETF), but the general rule has been that open specs please no one, slowly.

IMHO, FRAND and liberal cross-licensing produce better results.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#204
post #70

Earlier quoted context omitted.

You've got to remember that AMD are behind at all aspects of this, including documenting their work in an easily digestible way. "Support" means that the card is actively tested and presumably has some sort of SLA-style push to fix bugs for. As their stack matures, a bunch of cards that don't have official support will work well [0]. I have an unsupported card. There are horrible bugs. But the evidence I've seen is t…

CUDA dropped Tesla (from 2006!) only as of 7.0, which seems to have released around 2015. Fermi support lasted from 2010 until 2017, giving it a solid 7 years still. Kepler support was dropped around 2020, and the first cards were released in 2012. As such Fermi seems to be the shortest supported architecture, and it was around for 7 years. GCN4 (Polaris) was introduced in 2016, and seems to have been officially drop…

On top of this some Kepler support (for K80s etc) is still maintained in CUDA 11 which was last updated late 2022, and libraries like PyTorch and TensorFlow still support CUDA 11.8 out of the box.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#205

Earlier quoted context omitted.

With the most recent card being their one year old flagship ($1k) consumer GPU... Meanwhile CUDA supports anything with Nvidia stamped on it before it's even released. They'll even go as far as doing things like adding support for new GPUs/compute families to older CUDA versions (see Hopper/Ada and CUDA 11.8). You can go out and buy any Nvidia GPU the day of release, take it home, plug it in, and everything just work…

The most recent "card" is their MI300 line. It's annoying as hell to you and me that they are not catering to the market of people who want to run stuff on their gaming cards. But it's not clear it's bad strategy to focus on executing in the high-end first. They have been very successful landing MI300s in the HPC space... Edit: I just looked it up: 25% of the GPU Compute in the current Top500 Supercomputers is AMD ht…

I'm building a bare metal business around MI300x and top end Epyc CPUs. We will have them for rental soon. The goal is to build a public super computer that isn't just available to researchers in HPC.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#207
post #142

Earlier quoted context omitted.

> It has to play out long enough for the market to indicate a preference By what measures hasn't that happened already? CUDA been around and constantly improving for more than 15 years, and there is no competitors in sight so far. It's basically the de facto standard in many ecosystems.

There haven’t been any as successful, but there have been competitors. OpenCL, DirectX come to mind.

SYCL is the latest attempt that I'm aware of. It's still pretty active and may just work as it doesn't rely on video card manufactures to work out.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#208
post #34

Earlier quoted context omitted.

AMD is betting big on GPUs. They recently released the MI300, which has "2x transistors, 2.4x memory and 1.6x memory bandwidth more than the H100, the top-of-the-line artificial-intelligence chip made by Nvidia" ( https://www.economist.com/business/2024/01/31/could-amd-brea... ). They very much plan to compete in this space, and hope to ship $3.5B of these chips in the next year. Small compared to Nvidia's revenues o…

They are trying to compete in the segment of data center market where the shots are called by bean counters calculating FLOPS per dollar.

That's why I'm going to democratize that business and make it available to anyone who wants access. How does bare metal rentals of MI300x and top end Epyc CPUs sound? We take on the capex/opex/risk and give people what they want, which is access to HPC clusters.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#209
post #197

Earlier quoted context omitted.

The latest version of CUDA is 12.3, and version 12.2 came out 6 months prior. How many people are running an older version of CUDA right now on NVIDIA hardware for whatever particular reason? Even if AMD lagged support on CUDA versioning, I think it would be widely accepted if the performance per dollar at certain price points was better. Taking the whole market from NVIDIA is not really an option, it's better to att…

I just went through this this weekend - If you're running in Windows and want to use deepspeed, you have to still use Cuda 12.1 because deepspeed 13.1 is the latest that works with 12.1. There's no deepspeed for windows that works with 12.3. I tried to get it working this weekend but it was a huge PITA so I switched to putting everything into WSL2 then in arch on there pytorch etc in containers so I could flip versio…

Vent away! Sounds frustrating for sure.

As much as I love Microsoft/Windows for the work they have put into WSL, I ended up just putting Kubuntu on my devices and use QEMU with GPU passthrough whenever I need Windows. Gaming perf is good. You need an iGPU or a cheap second GPU for Linux in order to hand off a 4090 etc. to Windows (unless maybe your motherboard happens to support headless boot but if it's a consumer board it doesn't). Dual boot with Windows always gave me trouble.

Re: AMD funded a drop-in CUDA implementation built on ROCm: It's now open-source

#210

Earlier quoted context omitted.

They are focusing on HPC first. Which seems reasonable if your software stack is lacking. Look for sophisticated customers that can help build an ecosystem. As I mentioned elsewhere, 25% of GPU compute on the Top 500 Supercomputer list is AMD. This all on the back of a card that came out only three years ago. We are very rapidly moving towards a situation where there are many, many high-performance developers that wi…

Is a top 500 super computer list a good way of measuring relevancy in the future?

No, it isn't. What is a better measure is to look at businesses like what I'm building (and others), where we take on the capex/opex risk around top end AMD products and bring them to the masses through bare metal rentals. Previously, these sorts of cards were only available to the Top 500.
Post reply on HN