Anyone who has to work in this ecosystem surely thinks this is a naive take
AMD may get across the CUDA moat
81–90 of 312 posts
Re: AMD may get across the CUDA moat
#82Earlier quoted context omitted.
A simplistic economic take would suggest that the competition would result in lower prices, but given two players in the market who knows.
My intuition is along the lines that if AMD had a competing product earlier, then it would have kept prices down. But since Nvidia has shown what the market will pay, AMD won't be able to resist overcharging. It will probably come down a little, but nowhere near to the point of affordability. I sure hope I'm wrong.
AMD might have to charge less to break into customers that are already bought into Nvidia. There has to be a discount to cover the switching costs + still provide savings (or access).
Re: AMD may get across the CUDA moat
#83When coding using Vulkan, for graphics or compute (The latter is the relevant one here), you need to have CPU code (Written in C++, Rust etc), then serialize it as bytes, then have shaders which run on the graphics card. This 3-step process creates friction, much in the same way as backend/serialization/frontend does in web dev. Duplication of work, type checking not going across the bridge, the shader language being limited etc.
My understanding is CUDA's main strength is avoiding this. Do you agree? Is that why it's such a big deal? Ie, why this article was written, since you could always do compute shaders on AMD etc using Vulkan.
Re: AMD may get across the CUDA moat
#84Nope. PyTorch is not enough, you have to do come C++ occasionally (as the code there can be optimized radically, as we see in llama.cpp and the like). ROCm is unusable compared to CUDA (4x more code for the same problem).
I don't understand why everyone neglects good, usable and performant lower-level APIs. ROCm is fast, low-level, but much much harder to use than CUDA, and the market seems to agree.
Re: AMD may get across the CUDA moat
#85CUDA is the only reason I have an Nvidia card, but if more projects start migrating to a more agnostic environment, I'll be really grateful. Running Nvidia in Linux isn't as much fun. Fedora and Debian can be incredibly reliable systems, but when you add an Nvidia card, I feel like I am back in Windows Vista with kernel crashes from time to time.
I use a rolling distro (OpenSUSE Tumbleweed) and have had zero issues with my NVIDIA card despite it pulling the kernel and driver updates as they get released. The driver repo is maintained by NVIDIA itself, which is amazing.
Do you use wayland, multiple monitors, and/or play games or is it just for ML/AI?
Re: AMD may get across the CUDA moat
#86I was able to use ROCm recently with Pytorch and after pulling some hair it worked quite well. The Radeon GPU I had on hand was a bit old and underpowered (RDNA2) and it only supported matmul on fp64, but for the job I needed done I saw a 200x increase in it/s over CPU despite the need to cast everywhere, and that made me super happy. Best of all is that I simply set the device to `torch.device('cuda')` rather than o…
Sigh. It's great that these container images exist to give people an easy on-ramp, but they definitely don't work for every use case (especially once you're in embedded where space matters and you might not be online to pull multi-gb updates from some registry). So it's important that vendors don't feel let off the hook to provide sane packaging just because there's an option to use a kitchen-sink container image the…
> especially once you're in embedded
is this a real problem? exactly which embedded platform has a device that ROCm supports?
Re: AMD may get across the CUDA moat
#87Yup, thank the hobbyists. Pytorch is allowing other hardware. Stable diffusion working on m chips, intel arc, and Amd. Now what I'd like to see is real benchmarks for compute power. Might even get a few startups to compete in this new area.
It isn't the hobbyists who are making sure that PyTorch and other frameworks runs well on these chips, but teams of engineers who work for NVIDIA, AMD, Intel, etc. who are doing this as their primary assigned jobs, in exchange for money from their employer, who are paying those salaries because they want to sell chips into the enormous demand for running PyTorch faster. Hobbyist and open-source are definitely not syn…
People don't usually get employed to make things with no demand, and people who work for companies with a budget line don't really care how much the nVidia tax is. You can thank hobbyists for creating a lot of demand for compatability with other cards.
Re: AMD may get across the CUDA moat
#88There is only limited empirical evidence of AMD closing the gap that NVidia has created in the science or ML software. Even when considering pytorch only, the engineering effort to maintain specialized ROCm along with CUDA solutions is not trivial (think flashattention, or any customization that optimizes your own model). If your GPUs only need a simple ML workflow all times for a few years nonstop, maybe there exist…
The fact that El Capitan is AMD says that at least for Science/HPC there definitely is evidence of a closing gap.
Re: AMD may get across the CUDA moat
#89Earlier quoted context omitted.
Sigh. It's great that these container images exist to give people an easy on-ramp, but they definitely don't work for every use case (especially once you're in embedded where space matters and you might not be online to pull multi-gb updates from some registry). So it's important that vendors don't feel let off the hook to provide sane packaging just because there's an option to use a kitchen-sink container image the…
> especially once you're in embedded is this a real problem? exactly which embedded platform has a device that ROCm supports?
Robotic perception is the one relevant to me. You want to do object recognition on an industrial x86 or Jetson-type machine, without having to use Ubuntu or whatever the one "blessed" underlay system is (either natively or implicitly because you pulled a container based on it).
Re: AMD may get across the CUDA moat
#90Earlier quoted context omitted.
I never had an issue with nVidia drivers on Linux in the past 5 years, but recently bought a laptop with a 4090 and AMD CPU. Now I get random freezes, often right after I login into Cinnamon but can't really tell if it's the nVidia driver for 4090, AMDGPU driver for integrated RDNA, kernel 6.2 or Cinnamon issue. The laptop just hangs and stops responding to keyboard so I can't login to console and dmesg it.
The main issue with Nvidia on Linux AIUI is that they don't release the source code for their drivers.
That might be a philosophical problem that never prevented me from training models on Linux. The half-baked half-crashing AMD solutions just lead to wasting time I can spend on ML research instead.