AMD may get across the CUDA moat

Post by 65a » Sat, Oct 07, 2023, 5:15 AM UTC

arcanus wrote: post #179

withwarmup wrote: post #145

CUDA is the result of years of NVIDIA supporting the ecosystem, some people likes to complain because they bought hardware that was cheaper but can't use it for what they want to use it, when you buy NVIDIA, you aren't buying only the hardware, but the insane amount of work they have put into the ecosystem, the same goes for Intel, mkl and scikit-learn intelex aren't free to develop. AMD has the hardware but the supp…

> AMD has the hardware but the support for HPC is non-existent outside of the joke that is bliss and AOCL. You are probably two years behind the state of the art. The world's largest supercomputer, OLCF's Frontier, runs AMD CPUs and GPUs. It's emphatically using ROCm, not just BLIS and AOCL. See for example: https://docs.olcf.ornl.gov/systems/frontier_user_guide.html That's hardly non-existent support for HPC.

Post by mr_toad » Sat, Oct 07, 2023, 5:22 AM UTC

kkielhofner wrote: post #187

LtWorf wrote: post #141

Earlier quoted context omitted.

I've had kernel panics that disappeared when I started using the on board intel graphics instead of the nvidia. Your statement makes no sense. It's like a smoker claiming that since he didn't die of lung cancer, smoke is 100% safe.

Describing kernel panics and general nightmare scenarios as the general course with Nvidia doesn’t make sense either. Nvidia has 80% market share of the discrete GPU desktop market and at least 90% market share of cloud/datacenter. Nvidia GPUs are used almost exclusively for every cloud powered AI service and to train virtually every ML model in existence. Almost always on Linux. Do you really think any of this would…

Post by 65a » Sat, Oct 07, 2023, 5:35 AM UTC

Havoc wrote: post #57

dylan604 wrote: post #29

Earlier quoted context omitted.

>If you can field a competitively priced consumer card if this unicorn were to show up, what's to say that all the non-consumers won't just scarf up these equally performant yet lower priced cards causing the supply-demand situation we're in now? the only difference would be a sudden supply of the expensive Nvidia cards that nobody wants because of their price.

>if this unicorn were to show up A unicorn like that showed up a couple hours ago. Someone posted a guide for getting llama to run on a 7900xtx https://old.reddit.com/r/LocalLLaMA/comments/170tghx/guide_i... It's still slow and janky but this really isn't that far away. I don't buy that AMD can't make this happen if they actually tried. Go on fiverr, get them to compile a list of top 100 people in the DIY LLM space,…

Post by bdowling » Sat, Oct 07, 2023, 5:41 AM UTC

upbeat_general wrote: post #198

This article doesn’t address the real challenge [in my mind]. Framework support is one thing, but what about the million standalone CUDA kernels that have been written, especially common in research. Nobody wants to spend time re-writing/porting those, especially when they probably don’t understand the low-level details in the first place. Not to mention, what is the plan for comprehensive framework support? I’ve exp…

Post by NavinF » Sat, Oct 07, 2023, 6:10 AM UTC

wyldfire wrote: post #104

omneity wrote: post #52

I was able to use ROCm recently with Pytorch and after pulling some hair it worked quite well. The Radeon GPU I had on hand was a bit old and underpowered (RDNA2) and it only supported matmul on fp64, but for the job I needed done I saw a 200x increase in it/s over CPU despite the need to cast everywhere, and that made me super happy. Best of all is that I simply set the device to `torch.device('cuda')` rather than o…

> Best of all is that I simply set the device to `torch.device('cuda')` rather than openCL, which does wonders for compatibility Man oh man where did we go wrong that cuda is the more compatible option over OpenCL?