Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PyTorch 1.8, with AMD ROCm support

github.com

Post reply on HN
Print view View original

61–70 of 147 posts

Re: PyTorch 1.8, with AMD ROCm support

#61

Post by qayxc »

sandGorgon wrote: post #57
qayxc wrote: post #55

Earlier quoted context omitted.

Might have to do with the fact that AMD just doesn't seem to have the resources (see the common complaints about their drivers' quality) to fully support every chip. Another reason is certainly that they simply don't need to - just like Intel's iGPU, people working with deep learning opt for discrete GPUs (either built-in or external), both just isn't an option (yet?) for M1-based systems. The audience would be a nic…

I dont believe so...and looks like neither does either Apple or Google/Tensorflow - https://developer.apple.com/documentation/mlcompute - https://blog.tensorflow.org/2020/11/accelerating-tensorflow-... For a more personal take on your answer - do consider the rest of world. For example, Ryzen is very popular in India. Discrete GPU are unaffordable for that college student who wants to train a non-English NLP model on…

How does that contradict my point - Apple has to support the M1, since their M1-based SoCs don't support dGPUs (yet), so the M1 is all there is.

Besides, Apple is the most valuable company in the world and has exactly the kind of resources AMD doesn't have.

> or example, Ryzen is very popular in India. Discrete GPU are unaffordable for that college student who wants to train a non-English NLP model on GPU.

Well Google Collab is free and there are many affordable cloud-based offers as well. Training big DL models is not something you'd want to do on a laptop anyway. Small models can be trained on CPUs, so that shouldn't be an issue.

Inference is fast enough on CPUs anyway and if you really need to train models on your Ryzen APU, there's always other libraries, such as TensorflowJS, which is hardware agnostic as it runs on top of WebGL.

That's why I don't think this is a big deal at all - especially given that Intel still holds >90% of the integrated GPU market and doesn't even have an equivalent to ROCm. Again, niche within a niche, no matter where you look.

Re: PyTorch 1.8, with AMD ROCm support

#62

Post by meragrin_ »

carlmr wrote: post #38
cranium wrote: post #13

Earlier quoted context omitted.

It's the last thing that keeps me on Nvidia with proprietary Linux drivers. I wouldn't mind ML training on a AMD card to be slower but I need my workload to be at least GPU-accelerated.

I mean I wouldn't worry too much about it, I think if something big like PyTorch supports it AMD might rethink their strategy here. They have a lot to gain by entering the compute market.

AMD only cares about data center. Anything below that, they don't care. They don't care about supporting anything other than linux. God forbid some person try to experiment with their hardware/software as a hobby before putting it to use in a work setting.

Re: PyTorch 1.8, with AMD ROCm support

#63

Post by thecleaner »

std_badalloc wrote: post #44

PyTorch is the most impressive piece of software engineering that I know of. So yeah, it's a nice interface for writing fast numerical code. And for zero effort you can change between running on CPUs, GPUs and TPUs. There's some compiler functionality in there for kernel fusing and more. Oh, and you can autodiff everything. There's just an incredible amount of complexity being hidden behind behind a very simple inter…

Its python wrappers on top of existing ThTensor library which was already provided by torch. But yes great engineering nonetheless.

Re: PyTorch 1.8, with AMD ROCm support

#64

Post by vetinari »

dogma1138 wrote: post #29
mastazi wrote: post #8

Now that major frameworks finally started supporting ROCm, AMD has half-abandoned it (IIRC the last consumer cards supported were the Vega ones, cards from 2 generations ago). I hope this will change.

It’s worse.... It’s Linux only so no Mac, Windows or WSL. No support what so ever for APUs which means if you have a laptop without a dedicated GPU you’re out of luck (tho discrete mobile GPUs aren’t officially supported either and often do not work). They’ve not only haven’t been supporting any of their consumer based R“we promise it’s not GCN this time”DNA GPUs, but since December last year (2020) they’ve dropped s…

> It’s worse....

> It’s Linux only so no Mac, Windows or WSL.

I don't see really a problem here. You want Office or Photoshop? It runs on Mac and Windows only, so you better get one. You want ROCm? Get Linux for exactly the same reason.

Re: PyTorch 1.8, with AMD ROCm support

#65

Post by carlmr »

meragrin_ wrote: post #62
carlmr wrote: post #38

Earlier quoted context omitted.

I mean I wouldn't worry too much about it, I think if something big like PyTorch supports it AMD might rethink their strategy here. They have a lot to gain by entering the compute market.

AMD only cares about data center. Anything below that, they don't care. They don't care about supporting anything other than linux. God forbid some person try to experiment with their hardware/software as a hobby before putting it to use in a work setting.

Eh, not sure if that's correct. Their Ryzen consumer CPUs work amazingly well on Windows. The gaming graphics cards also still target Windows.

And if they do care about the data center that much and ROCm becomes a thing for compute, they will want as many people as possible to be able to experiment with ROCm at home. So that they demand data centers with ROCm.

Re: PyTorch 1.8, with AMD ROCm support

#66

Post by simcop2387 »

rfoo wrote: post #40
dheera wrote: post #37

Earlier quoted context omitted.

Jetson Nano: 1.4 TOPS/W, Coral TPU: 2 TOPS/W ? Of course it doesn't really help that Google refuses to release a more powerful TPU that can compete with e.g. Xavier NX or a V100 or RTX3080 so for lots of applications there isn't much of a choice but to use NVIDIA.

Sorry, should have mentioned "if you have access to Shenzhen" in my post :) What I have in mind is something like RK3399Pro, it has a proprietary NPU at roughly 3Tops / 1.5W (on paper). But its toolchain is rather hard to use. Hisilicon had similar offerings. There are also Kendryte K210 which claims 1Tops @ 0.3W but I haven't get any chance to try it. I was already playing with RK3399Pro When Edge TPU was announced,…

There's now a blackbox compiler that doesn't have to run on their service, but it's basically the same as all the others now because of that.

Re: PyTorch 1.8, with AMD ROCm support

#67

Post by vetinari »

slavik81 wrote: post #26
mastazi wrote: post #8

Now that major frameworks finally started supporting ROCm, AMD has half-abandoned it (IIRC the last consumer cards supported were the Vega ones, cards from 2 generations ago). I hope this will change.

I work for AMD, but this comment contains exclusively my personal opinions and information that is publicly available. ROCm has not been abandoned. PyTorch is built on top of rocBLAS, rocFFT, and Tensile (among other libraries) which are all under active development. You can watch the commits roll in day-by-day on their public GitHub repositories. I can't speak about hardware support beyond what's written in the docs…

I have no affiliation with AMD. Just own few pieces of their hardware.

The following is my speculation. For what I noticed, Polaris and Vega had a brute force approach; you could throw a workload on them and they would brute force through it. Navi generation is more gamer oriented; it does not have the hardware for the brutal approach, it relies more on optimizations from the software running on it and using it in specific ways. Provides better perfomance for games, but worse compute - and that's why it is not and is not going to be supported by ROCm.

The downside of course is, that you can no longer buy Vega. If I knew it last time it was on sale...

Re: PyTorch 1.8, with AMD ROCm support

#68

Post by my123 »

simcop2387 wrote: post #66
rfoo wrote: post #40

Earlier quoted context omitted.

Sorry, should have mentioned "if you have access to Shenzhen" in my post :) What I have in mind is something like RK3399Pro, it has a proprietary NPU at roughly 3Tops / 1.5W (on paper). But its toolchain is rather hard to use. Hisilicon had similar offerings. There are also Kendryte K210 which claims 1Tops @ 0.3W but I haven't get any chance to try it. I was already playing with RK3399Pro When Edge TPU was announced,…

There's now a blackbox compiler that doesn't have to run on their service, but it's basically the same as all the others now because of that.

On Xavier, the dedicated AI inference block is open source hardware.

Available at http://nvdla.org/

Re: PyTorch 1.8, with AMD ROCm support

#69

Post by jampekka »

std_badalloc wrote: post #44

PyTorch is the most impressive piece of software engineering that I know of. So yeah, it's a nice interface for writing fast numerical code. And for zero effort you can change between running on CPUs, GPUs and TPUs. There's some compiler functionality in there for kernel fusing and more. Oh, and you can autodiff everything. There's just an incredible amount of complexity being hidden behind behind a very simple inter…

OTOH PyTorch seems to be highly explosive if you try to use it outside the mainstream use (i.e. neural networks). There's sadly no performant autodiff system for general purpose Python. Numba is fine for performance, but does not support autodiff. JAX aims to be sort of general purpose, but in practice it is quite explosive when doing something other than neural networks.

A lot of this is probably due to supporting CPUs and GPUs with the same interface. There are quite profound differences in how CPUs and GPUs are programmed, so the interface tends to restrict especially more "CPU-oriented" approaches.

I have nothing against supporting GPUs (although I think their use is overrated and most people would do fine with CPUs), but Python really needs a general purpose, high performance autodiff.

Re: PyTorch 1.8, with AMD ROCm support

#70

Post by ahendriksen »

jampekka wrote: post #69
std_badalloc wrote: post #44

PyTorch is the most impressive piece of software engineering that I know of. So yeah, it's a nice interface for writing fast numerical code. And for zero effort you can change between running on CPUs, GPUs and TPUs. There's some compiler functionality in there for kernel fusing and more. Oh, and you can autodiff everything. There's just an incredible amount of complexity being hidden behind behind a very simple inter…

OTOH PyTorch seems to be highly explosive if you try to use it outside the mainstream use (i.e. neural networks). There's sadly no performant autodiff system for general purpose Python. Numba is fine for performance, but does not support autodiff. JAX aims to be sort of general purpose, but in practice it is quite explosive when doing something other than neural networks. A lot of this is probably due to supporting C…

What do you mean by “seems to be highly explosive”? I have used Pytorch to model many non-dnn things and have not experienced highly explosive behavior. (Could be that I have become too familiar with common footguns though)
Post reply on HN