Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD may get across the CUDA moat

hpcwire.com

Post reply on HN
Print view View original

131–140 of 312 posts

Re: AMD may get across the CUDA moat

#131

Post by distract8901 »

lhl wrote: post #119
HideousKojima wrote: post #113

Earlier quoted context omitted.

That was my experience, Nvidia Optimus (which is what allows dynamic switching between the integrated and dedicated GPU in laptops) was completely broken (as in a black screen, not just crashes or other issues) for several years, and Nvidia didn't care to do anything about it.

Yeah, Optimus was a huge PITA. I remember fighting with workarounds like bumblebee and prime for years. Also Nvidia dragged their feet on Wayland support for a few years too (and simultaneously was seemingly intent on sabotaging Nouveau).

I tried bumblebee again recently, and it works shockingly well now. I have a thinkpad T530 from 2013 with an NVS5400m.

There is some strange issue with some games where they don't get full performance from the dGPU, but more than the iGPU. I have to use optirun to get full performance.

It also has problems when the computer wakes from sleep. For whatever reason, hardware video decoding doesn't work after entering standby. Makes steam in home streaming crash on the client, but flipping to software decoding usually works fine.

The important part is that battery life is almost as good with bumblebee as it is with the dGPU turned off. No more fucking with Prime or rebooting into BIOS to turn the GPU back on.

Re: AMD may get across the CUDA moat

#132

Post by KeplerBoy »

omneity wrote: post #128
KeplerBoy wrote: post #108

Earlier quoted context omitted.

It must be a misnomer on PyTorch's side. Clearly it's neither CUDA nor OpenCL. AMD should just get it's shit together. This is ridiculous. Not the name, but the fact that you can only do FP64 on a GPU. Everybody is moving to FP16 and AMD is stuck on doubles?

I believe the fp64 limitation came from the laptop-grade GPU I had rather than inherent to AMD or ROCm. The API level I could target was at least two or three versions behind the latest they have to offer.

Might very well be true. I don't blame anyone for not diving deeper into figuring out why this stuff doesn't work.

But this is one of the great strengths of CUDA: I can develop a kernel on my workstation, my boss can demo it on his laptop and we can deploy it on Jetsons or the multi-gpu cluster with minimal changes and i can be sure that everything runs everywhere.

Re: AMD may get across the CUDA moat

#133

Post by kiratp »

Eisenstein wrote: post #87
mandevil wrote: post #55

Earlier quoted context omitted.

It isn't the hobbyists who are making sure that PyTorch and other frameworks runs well on these chips, but teams of engineers who work for NVIDIA, AMD, Intel, etc. who are doing this as their primary assigned jobs, in exchange for money from their employer, who are paying those salaries because they want to sell chips into the enormous demand for running PyTorch faster. Hobbyist and open-source are definitely not syn…

People don't usually get employed to make things with no demand, and people who work for companies with a budget line don't really care how much the nVidia tax is. You can thank hobbyists for creating a lot of demand for compatability with other cards.

There are so many billions of dollar being spent on this hardware that everyone other than Nvidia is doing everything they can to make competition happen.

Eg: https://www.intel.com/content/www/us/en/developer/videos/opt...

https://www.intel.com/content/www/us/en/developer/tools/onea...

https://developer.apple.com/metal/tensorflow-plugin/

Large scale opensource is, outside of a few exceptions, built by engineers paid to build it.

Re: AMD may get across the CUDA moat

#134

Post by einpoklum »

TL;DR:

1. Since PyTorch has grown very popular, and there's an AMD backend for that, one can switch GPU vendors when doing Generative AI work.

2. Like NVIDIA's Grace+Hopper CPU-GPU combo, AMD is/will be offering "Instinct MI300A", which improves performance over having the GPU across a PCIe bus from a regular CPU.

Re: AMD may get across the CUDA moat

#135

Post by einpoklum »

PH95VuimJjqBqy wrote: post #63
javchz wrote: post #26

CUDA is the only reason I have an Nvidia card, but if more projects start migrating to a more agnostic environment, I'll be really grateful. Running Nvidia in Linux isn't as much fun. Fedora and Debian can be incredibly reliable systems, but when you add an Nvidia card, I feel like I am back in Windows Vista with kernel crashes from time to time.

I see these complains from time to time and I never understand them. I've literally been running nvidia on linux since the TNT2 days and have _never_ had this sort of issue. That's across many drivers and many cards over the many many years.

I have been NVIDIA cards for compute capabilities only, both personally and at work, for nearly a decade. I've had dozens and dozens of different issues involving the hardware, the drivers, integration with the rest of the OS, version compatibilities, ensuring my desktop environment doesn't try to use the NVIDIA cards, etc. etc.

Having said that - I (or rarely, other people) have almost always managed to work out those issues and get my systems to work. Not in all cases though.

Re: AMD may get across the CUDA moat

#136

Post by mathisfun123 »

mikepurvis wrote: post #127
mathisfun123 wrote: post #124

Earlier quoted context omitted.

>industrial x86 or Jetson-type machine that's not embedded dev. if you 1. use underpowered devices to perform sophisticated tasks 2. using code/tools that operate at extremely high levels of "abstraction" don't be surprised when all the inherent complexity is tamed using just more layers of "abstraction". if that becomes a problem for your cost/power/space budget then reconsider choice 1 or choice 2.

Not sure this is worth an argument over semantics, but modern "embedded" development is a lot bigger than just microcontrollers and wearables. IMO as soon as you're deploying a computer into any kind of "appliance", or you're offline for periods of time, or you're running on batteries or your primary network connection is wireless... then yeah, you're starting to hit the requirements associated with embedded and need…

> IMO as soon as you're deploying a computer into any kind of "appliance", or you're offline for periods of time, or you're running on batteries or your primary network connection is wireless

yes and in those instances you do not reach for pytorch/tensorflow on top of ubuntu on top of x86 with a discrete gpu and 32gb of ram. instead you reach for C and micro or some arm soc that supports baremetal or at most rtos. that's embedded dev.

so i'll repeat myself: if you want to run extremely high-level code then don't be "surprised pikachu" when your underpowered platform, that you chose due to concrete, tight budgets doesn't work out.

Re: AMD may get across the CUDA moat

#137

Post by einpoklum »

nabla9 wrote: post #28

> Crossing the CUDA moat for AMD GPUs may be as easy as using PyTorch. Nvidia has spent huge amount of work to make code run smoothly and fast. AMD has to work hard to catch up. ROCm code is slower , has more bugs, don't have enough features and they have compatibility issues between cards.

> Nvidia has spent huge amount of work to make code run smoothly and fast.

Well, let's say "smoother" rather than "smoothly".

> ROCm code is slower

On physically-comparable hardware? Possible, but that's not an easy claim to make, certainly not as expansively as you have. References?

> has more bugs

Possible, but - NVIDIA keeps their bug database secret. I'm guessing you're concluding this from anecdotal experience? That's fair enough, but then - say so.

> ROCm ... don't have enough features and

Likely. while AMD has both spent less in that department (and had less to spend I guess); plus, and no less importantly - it tried to go along with the OpenCL initiative, as specified by the Khronos consortium, while NVIDIA has sort of "betrayed" the initiative by investing in it's vendor-locked, incompatible ecosystem and letting their OpenCL support decay in some respects.

> they have compatibility issues between cards.

such as?

Re: AMD may get across the CUDA moat

#138

Post by killerstorm »

zirgs wrote: post #107
hot_gril wrote: post #69

People complain about Nvidia being anticompetitive with CUDA, but I don't really see it. They saw a gap in the standards for on-GPU compute and put tons of effort into a proprietary alternative. They tied CUDA to their own hardware, which sorta makes technical sense given the optimizations involved, but it's their choice anyway. They still support the open standards, but many prefer CUDA and will pay the Nvidia premi…

CUDA was released in 2007 and the development of it started even earlier - possibly even in the 90s. Back then nobody else cared about GPU compute. OpenCL came out 2 years after that.

Not true. People got interested in general-purpose GPU compute (GPGPU) in early 2000s when video cards with programmable shaders became available. https://en.wikipedia.org/wiki/General-purpose_computing_on_g...

People made a programming language & a compiler/runtime for GPGPU in 2004: https://en.wikipedia.org/wiki/BrookGPU

Re: AMD may get across the CUDA moat

#139

Post by einpoklum »

pixelesque wrote: post #15

Does AMD have a solution to forward device combatibility (like PTX for NVidia)? Last time I looked into ROCm (two years ago?), you seemed to have to compile stuff explicitly for the architecture you were using, so if a new card came out, you couldn't use it without a recompile.

I don't know what they do with ROCm, but with OpenCL, the answer is: Certainly. It's called SPIR:

https://www.khronos.org/spir/

Re: AMD may get across the CUDA moat

#140

Post by mathisfun123 »

jauntywundrkind wrote: post #48
IronWolve wrote: post #2

Yup, thank the hobbyists. Pytorch is allowing other hardware. Stable diffusion working on m chips, intel arc, and Amd. Now what I'd like to see is real benchmarks for compute power. Might even get a few startups to compete in this new area.

Pytorch is just using Google's OpenXLA now, & OpenXLA is the actual cross platform thing, no? I'm not very well versed in this area, so pardon if mistaken. https://pytorch.org/blog/pytorch-2.0-xla-path-forward/

> Pytorch is just using Google's OpenXLA now

this is so far from accurate it should be considered libelous; from the link

> PyTorch/XLA is set to migrate to the open source OpenXLA

so PyTorch on the XLA backend is set to migrate to use OpenXLA instead of XLA. but basically everyone moved from XLA to OpenXLA because there is no more OSS XLA. so that's it. in general, PyTorch has several backends, including plenty of homegrown CUDA and CPU kernels. in fact the majority of your PyTorch code runs through PyTorch's own kernels.

Post reply on HN