PyTorch 1.8, with AMD ROCm support • Hacker News Board

Re: PyTorch 1.8, with AMD ROCm support

#91

Post by qayxc » Fri, Mar 05, 2021, 2:16 PM UTC

daniel-thompson wrote: post #73

qayxc wrote: post #55

Earlier quoted context omitted.

Might have to do with the fact that AMD just doesn't seem to have the resources (see the common complaints about their drivers' quality) to fully support every chip. Another reason is certainly that they simply don't need to - just like Intel's iGPU, people working with deep learning opt for discrete GPUs (either built-in or external), both just isn't an option (yet?) for M1-based systems. The audience would be a nic…

> AMD just doesn't seem to have the resources AMD's net income for 2020 was about $2.5B. If it was a management priority, they would fund more people to focus on this. I would love to support open-source drivers, but AMD's efforts with ROCm on consumer hardware are a joke. It's been said in other comments that AMD only cares about the datacenter. That certainly seems to be the case. So until AMD takes this seriously…

> So until AMD takes this seriously and gets a legitimate developer story together, I'm spending my money elsewhere.

Fair enough. Thing is, AMD's market share in the mobile market has been below 15% over the past years [1] and only last year increased to about 20%.

Of these 20%, how many notebooks are (think realistically for a second) intended to be used for DL while also not featuring an NVIDIA dGPU?

ROCm on consumer cards isn't a priority for AMD, since profits are small compared to the datacentre market and there's not that many people actually using consumer hardware for this kind of work.

I always feel there's a ton of bias going on and one should refer to sales data and market analysis to find out what the actual importance of one's particular niche really is.

AMD's focus w.r.t. consumer hardware is on gaming an CPU performance. That's just how it is and it's not going to change anytime soon. On the notebook side of things, and AMD APU + NVIDIA dGPU is the best you can get right now.

[1] https://www.tomshardware.com/news/amd-vs-intel-q3-2020-cpu-m...

Re: PyTorch 1.8, with AMD ROCm support

#92

Post by lumost » Fri, Mar 05, 2021, 2:31 PM UTC

Curious to see how this performs in a real world setting. My understanding is that Nvidia's neural network libs and other proprietary foo would still hold an edge over a standard AMD card.

If this is not the case then this is a really big deal.

Re: PyTorch 1.8, with AMD ROCm support

#93

Post by kortex » Fri, Mar 05, 2021, 2:31 PM UTC

vetinari wrote: post #86

sorenjan wrote: post #83

Earlier quoted context omitted.

> So ship linux version and let the customers decide whether they want it or not. What if I want to use GPGPU in Photoshop, or a game with more than two users? Or really anything aimed at consumers? > If you are doing server product That's irrelevant, server products can choose their own hardware and OS. > You can install Linux very easily "For a Linux user, you can already build such a system yourself quite triviall…

> What if I want to use GPGPU in Photoshop, or a game with more than two users? Or really anything aimed at consumers? Then use API supported on your target platform. It's not ROCm then. Maybe Vulcan Compute/DirectCompute/Metal Compute? > That's irrelevant, server products can choose their own hardware and OS. It is relevant for ROCm. > > You can install Linux very easily > "For a Linux user, you can already build su…

We all started as noobs once. A sizeable market of GPGPU is academics who don't yet have their nix chops. Had to walk an undergrad through basic git branching the other day.

Re: PyTorch 1.8, with AMD ROCm support

#94

Post by my123 » Fri, Mar 05, 2021, 2:33 PM UTC

vetinari wrote: post #86

sorenjan wrote: post #83

Earlier quoted context omitted.

> So ship linux version and let the customers decide whether they want it or not. What if I want to use GPGPU in Photoshop, or a game with more than two users? Or really anything aimed at consumers? > If you are doing server product That's irrelevant, server products can choose their own hardware and OS. > You can install Linux very easily "For a Linux user, you can already build such a system yourself quite triviall…

> What if I want to use GPGPU in Photoshop, or a game with more than two users? Or really anything aimed at consumers? Then use API supported on your target platform. It's not ROCm then. Maybe Vulcan Compute/DirectCompute/Metal Compute? > That's irrelevant, server products can choose their own hardware and OS. It is relevant for ROCm. > > You can install Linux very easily > "For a Linux user, you can already build su…

GPGPU with the graphics APIs isn’t a comparable developer experience at all, and comes with quite some major limitations.

About DirectCompute, C++ AMP is in practice dead on Windows, stuck at a DX11 feature level with no significant updates since 2012-13, staying present just for backwards compatibility.

Re: PyTorch 1.8, with AMD ROCm support

#95

Post by vetinari » Fri, Mar 05, 2021, 2:53 PM UTC

kortex wrote: post #93

vetinari wrote: post #86

Earlier quoted context omitted.

> What if I want to use GPGPU in Photoshop, or a game with more than two users? Or really anything aimed at consumers? Then use API supported on your target platform. It's not ROCm then. Maybe Vulcan Compute/DirectCompute/Metal Compute? > That's irrelevant, server products can choose their own hardware and OS. It is relevant for ROCm. > > You can install Linux very easily > "For a Linux user, you can already build su…

We all started as noobs once. A sizeable market of GPGPU is academics who don't yet have their nix chops. Had to walk an undergrad through basic git branching the other day.

I understand, but look at from another POV: ROCm is an USP of the Linux platform. Windows has its own USPs (DirectX or Office, for example), MacOS the same (iOS development, for example).

Why should Linux platform give up its competitive advantages against others? It would only diminish the reasons for running it. The other platforms won't do the same -- and nobody is bothered by that. In fact, it is generally considered to be an advantage of platform and reasons for getting it.

And it's not like getting Linux to run could be a significant expense (like purchasing an Apple computer is for many) or you need to get a license (like getting Windows). You can do it for free, and you will only learn something from it.

I've got my nix chops in the early '90 exactly this way: I wanted 32-bit flat-memory C compiler, didn't have the money for Watcom and associated DOS-extenders and djgpp wasn't a thing yet (or I didn't know about it yet). So I've got Slackware at home, as well as access to DG-UX at uni. It was different, I had to learn something, but ultimately, I'm glad I did.

Re: PyTorch 1.8, with AMD ROCm support

#96

Post by currymj » Fri, Mar 05, 2021, 3:01 PM UTC

ipsum2 wrote: post #28

buildbot wrote: post #12

It also looks like they are in the progress of adding Apple metal support, possibly for the M1: Part of 1.8 is this issue: https://github.com/pytorch/pytorch/pull/47635

What's the point? If you have enough money to buy a brand new Apple M1 laptop, you can afford a training rig or cloud credits. Any modern discrete GPU will blow away any M1 laptop for training. Is anyone training ML models on their ultra-thin laptop?

I use PyTorch for small-ish models, not really the typical enormous deep learning workload where you let it train for days.

I still prefer to work on a GPU workstation because it's the difference between running an experiment in minutes vs hours, makes it easier to iterate.

Some of that speed on a low-power laptop would be great. Much less friction.

Re: PyTorch 1.8, with AMD ROCm support

#97

Post by wing-_-nuts » Fri, Mar 05, 2021, 3:03 PM UTC

So, for someone not familiar, how far is AMD behind Nvidia's CUDA? I ask because AMD clearly has better linux driver support than Nvidia, and it would be awesome if their AI/ML libs were catching up.

Re: PyTorch 1.8, with AMD ROCm support

#98

Post by vetinari » Fri, Mar 05, 2021, 3:06 PM UTC

my123 wrote: post #94

vetinari wrote: post #86

Earlier quoted context omitted.

> What if I want to use GPGPU in Photoshop, or a game with more than two users? Or really anything aimed at consumers? Then use API supported on your target platform. It's not ROCm then. Maybe Vulcan Compute/DirectCompute/Metal Compute? > That's irrelevant, server products can choose their own hardware and OS. It is relevant for ROCm. > > You can install Linux very easily > "For a Linux user, you can already build su…

GPGPU with the graphics APIs isn’t a comparable developer experience at all, and comes with quite some major limitations. About DirectCompute, C++ AMP is in practice dead on Windows, stuck at a DX11 feature level with no significant updates since 2012-13, staying present just for backwards compatibility.

Then probably SYSCL is what you are looking for. But that one is still WIP.

Re: PyTorch 1.8, with AMD ROCm support

#99

Post by sandGorgon » Fri, Mar 05, 2021, 3:26 PM UTC

qayxc wrote: post #91

daniel-thompson wrote: post #73

Earlier quoted context omitted.

> AMD just doesn't seem to have the resources AMD's net income for 2020 was about $2.5B. If it was a management priority, they would fund more people to focus on this. I would love to support open-source drivers, but AMD's efforts with ROCm on consumer hardware are a joke. It's been said in other comments that AMD only cares about the datacenter. That certainly seems to be the case. So until AMD takes this seriously…

> So until AMD takes this seriously and gets a legitimate developer story together, I'm spending my money elsewhere. Fair enough. Thing is, AMD's market share in the mobile market has been below 15% over the past years [1] and only last year increased to about 20%. Of these 20%, how many notebooks are (think realistically for a second) intended to be used for DL while also not featuring an NVIDIA dGPU? ROCm on consum…

I disagree with you. AMD is running behind developers to use AMD for GPU based training. Just 1 month or so back, it announced partnership with AWS to get its GPU on the cloud.

https://aws.amazon.com/blogs/aws/new-amazon-ec2-g4ad-instanc...

So I would disagree with your claim about marketshare being the reason for its helplessness to create a superior developer-laptop experience.

Cluelessness? sure. But not helplessness. If it wants the developer market (versus the gamer), then it better start acting like a developer tools company...which includes cosying up to Google/Facebook/AWS/Microsoft and throwing money on ROCm. Education is one of them - https://developer.nvidia.com/educators/existing-courses ... and giving developers a generally superior development experience on the maximum number of machines is another.

Re: PyTorch 1.8, with AMD ROCm support

#100

Post by wxnx » Fri, Mar 05, 2021, 3:30 PM UTC

jampekka wrote: post #69

std_badalloc wrote: post #44

PyTorch is the most impressive piece of software engineering that I know of. So yeah, it's a nice interface for writing fast numerical code. And for zero effort you can change between running on CPUs, GPUs and TPUs. There's some compiler functionality in there for kernel fusing and more. Oh, and you can autodiff everything. There's just an incredible amount of complexity being hidden behind behind a very simple inter…

OTOH PyTorch seems to be highly explosive if you try to use it outside the mainstream use (i.e. neural networks). There's sadly no performant autodiff system for general purpose Python. Numba is fine for performance, but does not support autodiff. JAX aims to be sort of general purpose, but in practice it is quite explosive when doing something other than neural networks. A lot of this is probably due to supporting C…

> I have nothing against supporting GPUs (although I think their use is overrated and most people would do fine with CPUs), but Python really needs a general purpose, high performance autodiff.

As someone who works with machine learning models day-to-day (yes, some deep NNs, but also other stuff) - GPUs really seem unbeatable to me for anything gradient-optimization-of-matrices (i.e. like 80% of what I do) related. Even inference in a relatively simple image classification net takes an order of magnitude longer on CPU than GPU on the smallest dataset I'm working with.

Was this a comment about specific models that have a reputation as being more difficult to optimize on the GPU (like tree-based models - although Microsoft is working in this space)? Or am I genuinely missing some optimization techniques that might let me make more use of our CPU compute?