Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PyTorch 1.8, with AMD ROCm support

github.com

Post reply on HN
Print view View original

121–130 of 147 posts

Re: PyTorch 1.8, with AMD ROCm support

#121

Post by daniel-thompson »

qayxc wrote: post #115
daniel-thompson wrote: post #105

Earlier quoted context omitted.

> ROCm on consumer cards isn't a priority for AMD, since profits are small compared to the datacentre market and there's not that many people actually using consumer hardware for this kind of work. I think causality runs the other way: Profits are small and there aren't many people using AMD cards for this _because_ the developer experience for GPGPU on AMD is terrible (and that because it's not a priority for AMD).

That would imply that there even was a laptop market for AMD in the first place. As the market numbers show, up until last year, AMD simply wasn't relevant at all in the notebook segment, so what developer experience are you even talking about if there were no developers on AMD's platform?

I agree that AMD on mobile is a wasteland. But AMD has shipped over 500m desktop GPUs in the last 10 years. Surely some of those could/would have been better used for GPGPU dev if there was a decent developer experience.

Re: PyTorch 1.8, with AMD ROCm support

#122

Post by [deleted] »

slavik81 wrote: post #26
mastazi wrote: post #8

Now that major frameworks finally started supporting ROCm, AMD has half-abandoned it (IIRC the last consumer cards supported were the Vega ones, cards from 2 generations ago). I hope this will change.

I work for AMD, but this comment contains exclusively my personal opinions and information that is publicly available. ROCm has not been abandoned. PyTorch is built on top of rocBLAS, rocFFT, and Tensile (among other libraries) which are all under active development. You can watch the commits roll in day-by-day on their public GitHub repositories. I can't speak about hardware support beyond what's written in the docs…

[deleted]

Re: PyTorch 1.8, with AMD ROCm support

#123

Post by sumnuyungi »

rfoo wrote: post #39
tpetry wrote: post #34

Earlier quoted context omitted.

Because with support for M1 you can prototype your network on your local machine with „good“ performance. There are many cloud solutions etc. but for convenience nothing beats your local machine. You can use an IDE you like etc.

Because contrary to what you believe, M1 simply is not performant enough to be used to "prototype" your network. NNs can't be simply scaled up and down. It is *NOT* like those web apps which you can run on potatoes just fine as long as nobody are hitting them heavily.

This is false. You can prototype a network on an M1 [1] and teacher-student models are a de facto standard for scaling down.

You can trivially run transfer-learning on an M1 to prototype and see if a particular backbone fits well to a small dataset, then kickoff training on some cloud instance with the larger dataset for a few days.

[1] https://blog.tensorflow.org/2020/11/accelerating-tensorflow-...

Re: PyTorch 1.8, with AMD ROCm support

#124

Post by UncleOxidant »

std_badalloc wrote: post #44

PyTorch is the most impressive piece of software engineering that I know of. So yeah, it's a nice interface for writing fast numerical code. And for zero effort you can change between running on CPUs, GPUs and TPUs. There's some compiler functionality in there for kernel fusing and more. Oh, and you can autodiff everything. There's just an incredible amount of complexity being hidden behind behind a very simple inter…

> Oh, and you can autodiff everything.

Well, not everything. Julia's Zygote AD system can autodiff most Julia code (currently with the exception of code that mutates arrays/matrices).

Re: PyTorch 1.8, with AMD ROCm support

#125

Post by UncleOxidant »

jampekka wrote: post #69
std_badalloc wrote: post #44

PyTorch is the most impressive piece of software engineering that I know of. So yeah, it's a nice interface for writing fast numerical code. And for zero effort you can change between running on CPUs, GPUs and TPUs. There's some compiler functionality in there for kernel fusing and more. Oh, and you can autodiff everything. There's just an incredible amount of complexity being hidden behind behind a very simple inter…

OTOH PyTorch seems to be highly explosive if you try to use it outside the mainstream use (i.e. neural networks). There's sadly no performant autodiff system for general purpose Python. Numba is fine for performance, but does not support autodiff. JAX aims to be sort of general purpose, but in practice it is quite explosive when doing something other than neural networks. A lot of this is probably due to supporting C…

> There's sadly no performant autodiff system for general purpose Python.

Like there is for general purpose Julia code? (https://github.com/FluxML/Zygote.jl)

> I have nothing against supporting GPUs (although I think their use is overrated and most people would do fine with CPUs),

Do you run much machine learning code? All those matrix multiplications run a good bit faster on the GPU.

Re: PyTorch 1.8, with AMD ROCm support

#126

Post by codethief »

homarp wrote: post #58
dheera wrote: post #41

Earlier quoted context omitted.

I'll also add a caveat that toolage for Jetson boards is extremely incomplete. They supply you with a bunch of sorely outdated models for TensorRT like Inceptionv3 and SSD-MobileNetv2 and VGG-16. WTF, it's 2021. If you want to use anything remotely state-of-the-art like EfficientDet or HRNet or Deeplab or whatever you're left in the dark. Yes you can run TensorFlow or PyTorch (thankfully they give you wheels for thos…

https://github.com/wang-xinyu/tensorrtx has a lot of models implemented for TensorRT. They test on GTX1080 not jetson nano though, so some work is also needed. TVM is another alternative to get models to inference fast on nano

How does TVM compare to TensorRT performance-wise?

Re: PyTorch 1.8, with AMD ROCm support

#127

Post by microtonal »

jefft255 wrote: post #88
rfoo wrote: post #39

Earlier quoted context omitted.

Because contrary to what you believe, M1 simply is not performant enough to be used to "prototype" your network. NNs can't be simply scaled up and down. It is *NOT* like those web apps which you can run on potatoes just fine as long as nobody are hitting them heavily.

Not so sure about that. Here’s two things you can do (assuming you’re not training huge transformers or something). 1. Test your code with super low batch size. Bad for convergence, good for sanity check before submitting your job to a super computer. 2. Post-training evaluation. I’m pretty sure the M1 has enough power to do inference for not-so-big models. These two reasons are why I’m sometimes running stuff on my…

1. Test your code with super low batch size. Bad for convergence, good for sanity check before submitting your job to a super computer.

Or you can buy a desktop machine for the same price as an M1 MacBook with 32GB or 64GB RAM and an RTX2060 or RTX3060 (which support mixed-precision training) and you can actually finetune a reasonable transformer model with a reasonable batch size. E.g., I can finetune a multi-task XLM-RoBERTa base model just fine on an RTX2060, model distillation also works great.

Also, there are only so many sanity checks you can do on something as weak (when it comes to neural net training). Sure, you can check if your shapes are correct, loss is actually decreasing, etc. But once you get at the point your model is working, you will have to do dozens of tweaks that you can't reasonably do on an M1 and still want to do locally.

tl;dr: why make your life hard with an M1 for deep learning, if you can buy a beefy machine with a reasonable NVIDIA GPU at the same price? Especially if it is for work, your employer should just buy such a machine (and an M1 MacBook for on the go ;)).

Re: PyTorch 1.8, with AMD ROCm support

#128

Post by slavik81 »

microtonal wrote: post #108
wing-_-nuts wrote: post #97

So, for someone not familiar, how far is AMD behind Nvidia's CUDA? I ask because AMD clearly has better linux driver support than Nvidia, and it would be awesome if their AI/ML libs were catching up.

At my previous employer, we bought two Radeon VIIs (in addition to NVIDIA GPUs). The last time I tried it (just over ~6 months ago), there were still many bugs. Things would just crash and burn very frequently (odd shape errors, random crashes, etc.). Two colleagues reported some of those bugs in ROCm, but the bug reports were largely ignored. Maybe out-of-the-box support for PyTorch will result in more polish. Who k…

So, just up front: these are my personal opinions. I do not speak on behalf of AMD as a company. I'm just a software developer who works on ROCm. I joined AMD specifically because I wanted to help ROCm succeed.

If the problems you encountered are related to a particular ROCm software library, I would encourage you to open an issue on the library's GitHub page. You will get the best results if you can get your problem directly in front of the people responsible for fixing it. Radeon VII is one of the core supported platforms and bugs encountered with it should be taken seriously. In fact, if you want to point me at the bug reports, I will personally follow up on them.

ROCm has improved tremendously over the past 6 months, and I expect it to continue to improve. There have been growing pains, but I believe in the long-term success of this project.

Re: PyTorch 1.8, with AMD ROCm support

#129

Post by mastazi »

slavik81 wrote: post #26
mastazi wrote: post #8

Now that major frameworks finally started supporting ROCm, AMD has half-abandoned it (IIRC the last consumer cards supported were the Vega ones, cards from 2 generations ago). I hope this will change.

I work for AMD, but this comment contains exclusively my personal opinions and information that is publicly available. ROCm has not been abandoned. PyTorch is built on top of rocBLAS, rocFFT, and Tensile (among other libraries) which are all under active development. You can watch the commits roll in day-by-day on their public GitHub repositories. I can't speak about hardware support beyond what's written in the docs…

> PyTorch is built on top of rocBLAS, rocFFT, and Tensile (among other libraries) which are all under active development.

That's not very helpful if we can't use them on our own computers... Not many devs are able to put their hands on a datacentre-class card...

Re: PyTorch 1.8, with AMD ROCm support

#130

Post by slavik81 »

ckastner wrote: post #54
slavik81 wrote: post #26

Earlier quoted context omitted.

I work for AMD, but this comment contains exclusively my personal opinions and information that is publicly available. ROCm has not been abandoned. PyTorch is built on top of rocBLAS, rocFFT, and Tensile (among other libraries) which are all under active development. You can watch the commits roll in day-by-day on their public GitHub repositories. I can't speak about hardware support beyond what's written in the docs…

Hardware support is key, though. CUDA works with basically any card made in the last 5 years, consumer or compute. ROCm seems to work with a limited set of compute cards only. There's a ROCm team in Debian [1] trying to push ROCm forward (ROCm being open), but just getting supported hardware alone is already a major roadblock, which stalls the effort, and hence any contributions Debian could give back. [1] https://sa…

I didn't know about the Debian team. Thank you very much for informing me! I'm not sure how much I can do, but I would be happy to discuss with them the obstacles they are facing and see what I can help with. I'm better positioned to help with software issues than hardware matters, but I would love to hear about ROCm from their perspective. What's the best way to contact the team?
Post reply on HN