Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

PyTorch 1.8, with AMD ROCm support

github.com

Post reply on HN
Print view View original

141–147 of 147 posts

Re: PyTorch 1.8, with AMD ROCm support

#141

Post by t-vi »

rfoo wrote: post #36
etaioinshrdlu wrote: post #15

Earlier quoted context omitted.

Tensorflow doesn't seem to officially support ROCm, only unofficial community projects do. This is official support from PyTorch.

Tensorflow does officially support ROCm. The project was started by AMD and later upstreamed. https://github.com/tensorflow/tensorflow/tree/master/tensorf... https://github.com/tensorflow/tensorflow/blob/master/tensorf... It is true that it is not Google who are distributing binaries compiled with ROCm support through PyPI (tensorflow and tensorflow-gpu is uploaded by Google, but tensorflow-rocm is uploaded by AMD).…

What you describe sounds a lot like the PyTorch support before this announcement: You could download PyTorch from AMD's ROCm site or build it yourself for >= 2 years now and this worked very reliably. (Edit: The two years (Nov 2018 or so) are the ones I can attest to from using it personally, but it probably didn't start then.)

The news here is that the PyTorch team and AMD are confident enough about the quality that they're putting it on the front page. This has been a long way in the making, and finally achieving official support is a great step for the team working on it.

Re: PyTorch 1.8, with AMD ROCm support

#142

Post by t-vi »

mastazi wrote: post #8

Now that major frameworks finally started supporting ROCm, AMD has half-abandoned it (IIRC the last consumer cards supported were the Vega ones, cards from 2 generations ago). I hope this will change.

Part of the problem is that Vega GPUs are too good for crypto currency mining.

Example: I bought a 16GB Radeon VII for €550 (including 19% VAT). That would be a decent value proposition for Deep Learning even today. However, the card appears to trade second hand(!) for €1300+ on e-bay, because you apparently can run 90-100 MH/sec or so (this is an estimate I found on the web, I don't engage in this type of thing). As a result, you would not be able to get consumer Vega with these performance characteristics for a good price even if AMD produced them.

Re: PyTorch 1.8, with AMD ROCm support

#143

Post by codethief »

dheera wrote: post #138
codethief wrote: post #117

Earlier quoted context omitted.

TF-TRT doesn't work nearly as well as pure TRT. On my Jetson Nano a 300x300 SSD-MobileNetV2 with 2 object classes runs at 5 FPS using TF, <10 FPS using TF-TRT and 30 FPS using TensorRT.

This. Try any recent network with TF-TRT and you'll find that memory is constantly being copied back and forth between TF and TRT components of the system every time it stumbles upon an operation not supported in TRT. As such I often got slower results with TF-TRT than just pure TF, and at most a marginal improvement, even though what TRT does is conceptually awesome from a deployment standpoint, and if it only suppo…

> even though what TRT does is conceptually awesome from a deployment standpoint

I thought the same until, earlier this week, I realized that if I convert a model to TensorRT and serialize it & store it in a file that file is specific to my device (i.e. my specific Jetson Nano), meaning that my colleagues can't run that file on their Jetson Nano. What the actual fuck.

Do you happen to have found a workaround for this? I really don't want to have to convert the model anew every single time I deploy it. There are just too many moving parts involved in the conversion process, dependency-wise.

Re: PyTorch 1.8, with AMD ROCm support

#144

Post by codethief »

dheera wrote: post #137
codethief wrote: post #120

Earlier quoted context omitted.

> I'll also add a caveat that toolage for Jetson boards is extremely incomplete. A hundred times this. I was about to write another rant here but I already did that[0] a while ago, so I'll save my breath this time. :) Another fun fact regarding toolage: Today I discovered that many USB cameras work poorly on Jetsons (at least when using OpenCV), probably due to different drivers and/or the fact that OpenCV doesn't su…

They provide some SSD-Mobilenet-v2 here: https://github.com/dusty-nv/jetson-inference Yeah, it works. I get 140 fps on a Xavier NX. It's super impressive for the wattage and size of the device. But they want you to train it using their horrid "DIGITS" interface, and it doesn't support any more recent networks. I really wish Nvidia would stop trying to reinvent the wheel in training and focus on keeping up with being…

> They provide some SSD-Mobilenet-v2 here: https://github.com/dusty-nv/jetson-inference

I was aware of that repository but from taking a cursory look at it I had thought dusty was just converting models from PyTorch to TensorRT, like here[0, 1]. Am I missing something? (EDIT: Oh, never mind. You probably meant the model trained on COCO[2]. Now I remember that I ignored it way back when because I needed much better accuracy.)

> I get 140 fps on a Xavier NX

That really is impressive. Holy shit.

[0]: https://github.com/dusty-nv/jetson-inference/blob/master/doc...

[1]: https://github.com/dusty-nv/jetson-inference/issues/896#issu...

[2]: https://github.com/dusty-nv/jetson-inference/blob/master/doc...

Re: PyTorch 1.8, with AMD ROCm support

#145

Post by slavik81 »

h0l0gr4ph1c wrote: post #136
slavik81 wrote: post #128

Earlier quoted context omitted.

So, just up front: these are my personal opinions. I do not speak on behalf of AMD as a company. I'm just a software developer who works on ROCm. I joined AMD specifically because I wanted to help ROCm succeed. If the problems you encountered are related to a particular ROCm software library, I would encourage you to open an issue on the library's GitHub page. You will get the best results if you can get your problem…

As someone who bought a RX580 for playing with Deep learning with ROCm (It was supported at the time). After posting to one or two bug threads, I had the same experience as the gp -- our issues were ignored. The issues have recently closed as the RX580 is no longer supported. As for long term success, good luck, but once bitten twice shy.

I empathize. There have been plenty of mistakes. Trust is difficult to earn, and clearly we have not acted in a manner deserving of yours.

When I first encountered ROCm, my impressions were mixed. The idea thrilled me, but the execution did not. I planned to ignore ROCm until it was clear that it would meet my needs. Obviously, my plans changed, but I haven't forgotten the perspective I had as a potential user.

There are still rough edges and I know that nobody gets third chances, so it's good that you're cautious. We are steadily improving and I believe we will do better in supporting our users going forward, but to earn that trust back, we will have to prove it through our actions.

Re: PyTorch 1.8, with AMD ROCm support

#146

Post by homarp »

codethief wrote: post #126
homarp wrote: post #58

Earlier quoted context omitted.

https://github.com/wang-xinyu/tensorrtx has a lot of models implemented for TensorRT. They test on GTX1080 not jetson nano though, so some work is also needed. TVM is another alternative to get models to inference fast on nano

How does TVM compare to TensorRT performance-wise?

it depends of your model

https://forums.developer.nvidia.com/t/tensorrt-vs-tvm/79964

https://discuss.tvm.apache.org/t/which-is-better-tvm-or-tens...

Re: PyTorch 1.8, with AMD ROCm support

#147

Post by dkersten »

ebegnwgnen wrote: post #60
slavik81 wrote: post #26

Earlier quoted context omitted.

I work for AMD, but this comment contains exclusively my personal opinions and information that is publicly available. ROCm has not been abandoned. PyTorch is built on top of rocBLAS, rocFFT, and Tensile (among other libraries) which are all under active development. You can watch the commits roll in day-by-day on their public GitHub repositories. I can't speak about hardware support beyond what's written in the docs…

"Future plan"... it's been years and it's still "future plan". I want to buy AMD because they are more open than Nvidia. But Nvidia supports CUDA day one for all their graphic cards and AMD still don't have rocm support on most of their product even years after their release [0] Given AMD size & budget, the reason why they don't hire a few more employee full time on making rocm work with their own graphic card is bey…

I really don't want to buy Nvidia, but AMD really isn't an alternative if you do anything outside of gaming...
Post reply on HN