Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PyTorch 1.8, with AMD ROCm support

github.com

Post reply on HN
Print view View original

81–90 of 147 posts

Re: PyTorch 1.8, with AMD ROCm support

#81

Post by jl2718 »

jampekka wrote: post #69
std_badalloc wrote: post #44

PyTorch is the most impressive piece of software engineering that I know of. So yeah, it's a nice interface for writing fast numerical code. And for zero effort you can change between running on CPUs, GPUs and TPUs. There's some compiler functionality in there for kernel fusing and more. Oh, and you can autodiff everything. There's just an incredible amount of complexity being hidden behind behind a very simple inter…

OTOH PyTorch seems to be highly explosive if you try to use it outside the mainstream use (i.e. neural networks). There's sadly no performant autodiff system for general purpose Python. Numba is fine for performance, but does not support autodiff. JAX aims to be sort of general purpose, but in practice it is quite explosive when doing something other than neural networks. A lot of this is probably due to supporting C…

Have you tried using Enzyme* on Numba IR?

* https://enzyme.mit.edu

Re: PyTorch 1.8, with AMD ROCm support

#82

Post by tpetry »

rfoo wrote: post #39
tpetry wrote: post #34

Earlier quoted context omitted.

Because with support for M1 you can prototype your network on your local machine with „good“ performance. There are many cloud solutions etc. but for convenience nothing beats your local machine. You can use an IDE you like etc.

Because contrary to what you believe, M1 simply is not performant enough to be used to "prototype" your network. NNs can't be simply scaled up and down. It is *NOT* like those web apps which you can run on potatoes just fine as long as nobody are hitting them heavily.

You can just run your training with a lot less of data.

Re: PyTorch 1.8, with AMD ROCm support

#83

Post by sorenjan »

vetinari wrote: post #77
sorenjan wrote: post #72

Earlier quoted context omitted.

It's a big problem if you want to use GPGPU in a product shipped to customers, or if you want to do some GPU programming on your own computer in your free time. Keep in mind that not only is it Linux only, it doesn't work on consumer hardware either.

> It's a big problem if you want to use GPGPU in a product shipped to customers, So ship linux version and let the customers decide whether they want it or not. If you are doing server product, it is even handy; just ship docker image or docker-compose/set of k8s pods, if it needs them. > or if you want to do some GPU programming on your own computer in your free time. You can install Linux very easily; it's not that…

> So ship linux version and let the customers decide whether they want it or not.

What if I want to use GPGPU in Photoshop, or a game with more than two users? Or really anything aimed at consumers?

> If you are doing server product

That's irrelevant, server products can choose their own hardware and OS.

> You can install Linux very easily

"For a Linux user, you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem"

Also, their main competitor with a huge market lead have this: https://docs.nvidia.com/cuda/cuda-installation-guide-microso...

> It does, just not on the current gen one or even currently purchasable one. I agree, this is a problem.

Only on old hardware, and you can't expect users' computers to be compatible. In fact, you should expect users' copmuters to be incompatible. That's like saying consumer hardware support GLIDE.

Re: PyTorch 1.8, with AMD ROCm support

#84

Post by dogma1138 »

vetinari wrote: post #64
dogma1138 wrote: post #29

Earlier quoted context omitted.

It’s worse.... It’s Linux only so no Mac, Windows or WSL. No support what so ever for APUs which means if you have a laptop without a dedicated GPU you’re out of luck (tho discrete mobile GPUs aren’t officially supported either and often do not work). They’ve not only haven’t been supporting any of their consumer based R“we promise it’s not GCN this time”DNA GPUs, but since December last year (2020) they’ve dropped s…

> It’s worse.... > It’s Linux only so no Mac, Windows or WSL. I don't see really a problem here. You want Office or Photoshop? It runs on Mac and Windows only, so you better get one. You want ROCm? Get Linux for exactly the same reason.

No I don’t want ROCm I want GPGPU, which is why the entire planet is running CUDA.

Every decision AMD made with ROCm seems to boil down too “i never want to be actually competitive with CUDA”.

Everything from being Linux only and yes that’s is a huge huge huge limitation because most people don’t want ROCm but many would like Photoshop or Premier or Blender being able to use it. Especially with the OpenCL issues AMD has....

Through them not supporting their own hardware to other mind boggling decisions like the fact that ROCm binaries are hardware specific so you have no guarantee of interoperability and worse no guarantee for future compatibility and in fact it does break.

The point is that I can take a 6 years old CUDA binary and run it today still on everything form an embedded system through a high end server, it would maintain its compatibility across multiple generations of GPU hardware, multi operating systems and multiple CPU architectures.

And if you can’t understand why that is not only valuable but more or less mandatory for every GPGPU application you would ever ship to customers, perhaps you should stick to photoshop.

AMD is a joke in this space and they are very much actively continuing in making themselves one with every decision they make.

It’s not that they’ve abandoned ROCm is that it’s entire existence is telegraphing “Don't take use seriously, because we don’t”.

Re: PyTorch 1.8, with AMD ROCm support

#85

Post by rrss »

thecleaner wrote: post #63
std_badalloc wrote: post #44

PyTorch is the most impressive piece of software engineering that I know of. So yeah, it's a nice interface for writing fast numerical code. And for zero effort you can change between running on CPUs, GPUs and TPUs. There's some compiler functionality in there for kernel fusing and more. Oh, and you can autodiff everything. There's just an incredible amount of complexity being hidden behind behind a very simple inter…

Its python wrappers on top of existing ThTensor library which was already provided by torch. But yes great engineering nonetheless.

I don't think this is a particularly accurate description of pytorch in 2021. Yeah, the original c++ backend came from torch, but I think most of that has been replaced. AFAIK, all the development of the c++ backend for pytorch over that last several years has been done as part of the pytorch project -it's not just python wrappers at this point.

Re: PyTorch 1.8, with AMD ROCm support

#86

Post by vetinari »

sorenjan wrote: post #83
vetinari wrote: post #77

Earlier quoted context omitted.

> It's a big problem if you want to use GPGPU in a product shipped to customers, So ship linux version and let the customers decide whether they want it or not. If you are doing server product, it is even handy; just ship docker image or docker-compose/set of k8s pods, if it needs them. > or if you want to do some GPU programming on your own computer in your free time. You can install Linux very easily; it's not that…

> So ship linux version and let the customers decide whether they want it or not. What if I want to use GPGPU in Photoshop, or a game with more than two users? Or really anything aimed at consumers? > If you are doing server product That's irrelevant, server products can choose their own hardware and OS. > You can install Linux very easily "For a Linux user, you can already build such a system yourself quite triviall…

> What if I want to use GPGPU in Photoshop, or a game with more than two users? Or really anything aimed at consumers?

Then use API supported on your target platform. It's not ROCm then. Maybe Vulcan Compute/DirectCompute/Metal Compute?

> That's irrelevant, server products can choose their own hardware and OS.

It is relevant for ROCm.

> > You can install Linux very easily

> "For a Linux user, you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem"

I'm quite surprised to see comparisons ad absurdum after suggesting to use proper tool for a job. Installing a suitable operating system - for a supposed hacker - is a convoluted, nonsensical action nowadays?

> Only on old hardware, and you can't expect users' computers to be compatible. In fact, you should expect users' copmuters to be incompatible. That's like saying consumer hardware support GLIDE.

Vega is not that old; the problem is that is not not procurable anymore, which, if you read my comment again, I agreed that it is a problem.

Re: PyTorch 1.8, with AMD ROCm support

#87

Post by meragrin_ »

carlmr wrote: post #78
sorenjan wrote: post #76

Earlier quoted context omitted.

ROCm has been a thing for four years now, Cuda had been around almost a decade before ROCm was conceived, and AMD still doesn't show any interest in an accessible compute platform for all developers and users. I think they should focus on OpenCL and Sycl.

Well it's a chicken and egg problem. Without major library support nobody cares about ROCm, if nobody cares about ROCm AMD management will focus on other things. But PyTorch just laid that egg.

As far as ROCm support on consumer products, it is strictly a management problem which will takes years for them to figure out since they do not listen to their engineers and do not view users of consumer graphics cards as compute customers.

I would love an alternative to Nvidia cards. After waiting for so long for ROCm support for RDNA cards and reading an engineer's comments about why there is no support yet, I've given up on AMD for compute support on their graphics cards. I'm hoping Intel's graphic cards aren't garbage and get quick support. I probably will buy an Nvidia card before then if I have the opportunity since I'm tired of waiting for an alternative.

Re: PyTorch 1.8, with AMD ROCm support

#88

Post by jefft255 »

rfoo wrote: post #39
tpetry wrote: post #34

Earlier quoted context omitted.

Because with support for M1 you can prototype your network on your local machine with „good“ performance. There are many cloud solutions etc. but for convenience nothing beats your local machine. You can use an IDE you like etc.

Because contrary to what you believe, M1 simply is not performant enough to be used to "prototype" your network. NNs can't be simply scaled up and down. It is *NOT* like those web apps which you can run on potatoes just fine as long as nobody are hitting them heavily.

Not so sure about that. Here’s two things you can do (assuming you’re not training huge transformers or something).

1. Test your code with super low batch size. Bad for convergence, good for sanity check before submitting your job to a super computer.

2. Post-training evaluation. I’m pretty sure the M1 has enough power to do inference for not-so-big models.

These two reasons are why I’m sometimes running stuff on my own GTX 1060, even though it’s pretty anemic and I wouldn’t actually do a training run there.

There quite a bit of friction to training in the cloud, especially if it’s on shared cluster (which is what I have access to). You have a quota, and wait time when the supercomputer is under load. Sometimes you just need to quickly fire up something!

Re: PyTorch 1.8, with AMD ROCm support

#89

Post by jefft255 »

rsfern wrote: post #71
rfoo wrote: post #39

Earlier quoted context omitted.

Because contrary to what you believe, M1 simply is not performant enough to be used to "prototype" your network. NNs can't be simply scaled up and down. It is *NOT* like those web apps which you can run on potatoes just fine as long as nobody are hitting them heavily.

Sure you probably don’t want to do full training runs locally, but There’s a lot you can do locally that has a lot of added friction on a gpu cluster or other remote compute resource I like to start a new project by prototyping and debugging my training and cunning config code, setting up the data loading and evaluation pipeline, hacking around with some baseline models and making sure they can overfit some small sub…

Coding somewhat locally is a must for me too because the cluster I have access to has pretty serious wait times (up to a couple hours on busy days). Imagine only being able to run the code you’re writing a few times a day at most! Iterative development and doing a lot of mistakes is how I code; I don’t want to go back to punch card days where you waited and waited before you ended up with a silly error.

Re: PyTorch 1.8, with AMD ROCm support

#90

Post by sumnuyungi »

ipsum2 wrote: post #33
upbeat_general wrote: post #30

Earlier quoted context omitted.

I don’t understand this logic. If someone has $1000 for an entry level m1 machine then they also have enough money for a separate rig with a GPU that’s probably another $600-1000 for something decent? Cloud GPUs are also pretty expensive. I don’t think anyone is seriously training their ML models on their ultra thin laptop but I think the ability to do so would make it easier for lots of people to get started with wh…

If someone is shelling out for a brand new, early adopter product, then they probably have a decent amount of money. Even when TensorFlow and PyTorch implement training support on the M1, it will be useless for practically anything except training 2-3 layer models on MNIST. So why should valuable engineering time be spent on this?

This is just patently false. Most of the folks I know that have an M1 are students that were saving up to upgrade from a much older computer and got the M1 Air. I can assure you that they don't have a decent amount of money.

Tensorflow has a branch that's optimized for metal with impressive performance. [1] It's fast enough to do transfer learning quickly on a large resnet, which is a common use-case for photo/video editing apps that have ML-powered workflows. It's best for everyone to do this locally: maintains privacy for the user and eliminates cloud costs for the developer.

Also, not everyone has an imagenet sized dataset. A lot of applied ML uses small networks where prototyping is doable on a local machine.

[1] https://blog.tensorflow.org/2020/11/accelerating-tensorflow-...

Post reply on HN