Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

AMD may get across the CUDA moat

hpcwire.com

Post reply on HN
Print view View original

71–80 of 312 posts

Re: AMD may get across the CUDA moat

#71

Post by kombine »

javchz wrote: post #26

CUDA is the only reason I have an Nvidia card, but if more projects start migrating to a more agnostic environment, I'll be really grateful. Running Nvidia in Linux isn't as much fun. Fedora and Debian can be incredibly reliable systems, but when you add an Nvidia card, I feel like I am back in Windows Vista with kernel crashes from time to time.

I use a rolling distro (OpenSUSE Tumbleweed) and have had zero issues with my NVIDIA card despite it pulling the kernel and driver updates as they get released. The driver repo is maintained by NVIDIA itself, which is amazing.

Re: AMD may get across the CUDA moat

#72

Post by pama »

There is only limited empirical evidence of AMD closing the gap that NVidia has created in the science or ML software. Even when considering pytorch only, the engineering effort to maintain specialized ROCm along with CUDA solutions is not trivial (think flashattention, or any customization that optimizes your own model). If your GPUs only need a simple ML workflow all times for a few years nonstop, maybe there exist corner cases where the finances make sense. It is hard for AMD now to close the gap across the scientific/industrial software base of CUDA. NVidia feels like a software company for the hardware they produce; luckily they make the money from hardware thus cannot lock the software libraries.

(Edited “no” to limited empirical evidence after a fellow user mentioned El Capitan.)

Re: AMD may get across the CUDA moat

#73

Post by fwsgonzo »

mikepurvis wrote: post #67
omneity wrote: post #52

I was able to use ROCm recently with Pytorch and after pulling some hair it worked quite well. The Radeon GPU I had on hand was a bit old and underpowered (RDNA2) and it only supported matmul on fp64, but for the job I needed done I saw a 200x increase in it/s over CPU despite the need to cast everywhere, and that made me super happy. Best of all is that I simply set the device to `torch.device('cuda')` rather than o…

Sigh. It's great that these container images exist to give people an easy on-ramp, but they definitely don't work for every use case (especially once you're in embedded where space matters and you might not be online to pull multi-gb updates from some registry). So it's important that vendors don't feel let off the hook to provide sane packaging just because there's an option to use a kitchen-sink container image the…

I feel the same way, especially about build systems. OpenSSL and v8 are among a large list of things that have horrid build systems. Only way to build them sanely is to use some randos CMake fork, then it Just Works. Literally a two-liner in your build system to add them to your project with a sane CMake script.

Re: AMD may get across the CUDA moat

#74

Post by redeeman »

tibbydudeza wrote: post #43
sumtechguy wrote: post #20

Earlier quoted context omitted.

It is oligopoly pricing. https://www.investopedia.com/terms/o/oligopoly.asp With that few competitors pricing would not change much.

Price difference between 13900K and AMD Ryzen 9 7950x is not big - the latest 7950X3D is about on par with the higher clocked 13900KS as well.

because intel lowered their prices

Re: AMD may get across the CUDA moat

#75

Post by rottencupcakes »

latchkey wrote: post #66
frnkng wrote: post #62

As a former ETH miner I learned the hard way that saving a few bucks on hardware may not be worth operational issues. I had a miner running with Nividia cards and a miner running with AMD cards. One of them had massive maintenance demand and the other did not. I will not state which brand was better imho. Currently I estimate that running miners and running gpu servers has similar operational requirements and finally…

I ran 150,000+ AMD cards for mining ETH. Once I fully automated all the vbios installs and individual card tuning, it ran beautifully. Took a lot of work to get there though! Fact is that every single GPU chip is a snowflake. No two operate the same.

Have you ever written about this enterprise? This sounds super unique and I would be very interested in hearing about how it was run and how it turned out.

Re: AMD may get across the CUDA moat

#76

Post by mikepurvis »

fwsgonzo wrote: post #73
mikepurvis wrote: post #67

Earlier quoted context omitted.

Sigh. It's great that these container images exist to give people an easy on-ramp, but they definitely don't work for every use case (especially once you're in embedded where space matters and you might not be online to pull multi-gb updates from some registry). So it's important that vendors don't feel let off the hook to provide sane packaging just because there's an option to use a kitchen-sink container image the…

I feel the same way, especially about build systems. OpenSSL and v8 are among a large list of things that have horrid build systems. Only way to build them sanely is to use some randos CMake fork, then it Just Works. Literally a two-liner in your build system to add them to your project with a sane CMake script.

I was part of a Nix migration over the past two years, and literally one of the first things we checked is that there was already a community-maintained tensorflow+gpu package in nixpkgs because without that the whole thing would have been a complete non-starter, and we sure as heck didn't have the resources or know-how to figure it out for ourselves as a small DevOps team just trying to do basic packaging.

Re: AMD may get across the CUDA moat

#77

Post by bigcat12345678 »

mikepurvis wrote: post #70
bigcat12345678 wrote: post #44

Cuda is the foundation NVIDIA moat is the years of work built by oss community, big corporations, research insistute They spend all time building for cuda, a lot of implicit designs are derived from cuda's characteristic That will be the main challenge

It depends on the domain. Increasingly people's interfaces to this stuff are the higher level libraries like tensorflow, pytorch, numpy/cupy, and to a lesser degree accelerated processing libraries such as opencv, PCL, suitesparse, ceres-solver, and friends. If you can add hardware support to a major library and improve on the packaging and deployment front while also undercutting on price, that's the moat gone overn…

what I meant is that all these stuff have 15 years of implicit accumulation of knowledge and tips and even hacks builtin in the software

No matter what you depends on, you'll have a slew of larger or minor obstacles or annoyance

That collectively is the most itself

As you said, already it's clear that replacing cuda itself is not that daunting

Re: AMD may get across the CUDA moat

#78

Post by slavik81 »

tomsmeding wrote: post #58
ginko wrote: post #23

Earlier quoted context omitted.

When I try to install rocm-ml-sdk on Arch linux it'll tell me the total installed size would be about 18GB. What can possibly explain this much bloat for what should essentially be a library on top of a graphics driver as well as some tools (compiler, profiler etc.)? A couple hundred MB I could understand if they come with graphical apps and demos, but not this..

A regular TensorFlow installation, just the Python library, is an 184 MB wheel that unpacks to about 1.2 GB of stuff. I have no clue what mess goes in there, but it's a lot. Still, if you're right that this package seems to take 18 GB disk size, something weird is going on.

There's a lot of kernels that are specialized for particular sets of input parameters and tuned for improved performance on specific hardware, which makes the libraries a couple hundred megabytes per architecture. The ROCm libraries are huge because they are fat binaries containing native machine code for ~13 different GPU architectures.

Re: AMD may get across the CUDA moat

#79

Post by kombine »

smoldesu wrote: post #35
javchz wrote: post #26

CUDA is the only reason I have an Nvidia card, but if more projects start migrating to a more agnostic environment, I'll be really grateful. Running Nvidia in Linux isn't as much fun. Fedora and Debian can be incredibly reliable systems, but when you add an Nvidia card, I feel like I am back in Windows Vista with kernel crashes from time to time.

Those problems might just be GNOME-related at this point. I've been daily-driving two different Nvidia cards for ~3 years now (1050 Ti then 3070 Ti) and Wayland has felt pretty stable for the past 12 months. The worst problem I had experienced in that time was Electron and Java apps drawing incorrectly in xWayland, but both of those are fixed upstream. I'm definitely not against better hardware support for AI, but I…

I really hope that with KDE 6 I can finally switch to Wayland!

Re: AMD may get across the CUDA moat

#80

Post by hot_gril »

fluxem wrote: post #65

I call it the 90% problem. If AMD works for 90% of my projects, I would still buy NVIDIA, which works for 100%, even though I’m paying a premium

I'm lazy, so it's 99% for me. I don't even mess with AMD CPUs; I know they're not exactly the same instruction set as Intel, and more importantly they work with a different (and less mainstream) set of mobos, so I don't want em. If AMD manages to pull more customers their way, that's great, it just means lower Intel premium for me.
Post reply on HN