Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accelerated PyTorch Training on M1 Mac

pytorch.org

Post reply on HN
Print view View original

31–40 of 153 posts

Re: Accelerated PyTorch Training on M1 Mac

#31

Post by LeanderK »

my123 wrote: post #11
Scene_Cast2 wrote: post #8

I'm curious about the performance compared to something like, say, the RTX 3070.

Low. Apple doesn't have matrix math accelerators in their current GPUs. The neural engine is small and inference only. It's also only exposed by a far higher level interface, CoreML. Where it could still make sense is if you have a small VRAM pool on the dGPU and a big one on the M1, but with the price of a Mac, not sure that makes a lot of sense either in most scenarios compared to paying for a big dGPU.

> The neural engine is small and inference only

Why is it inference only? At least the operations are the same...just a bunch of linear algebra

Re: Accelerated PyTorch Training on M1 Mac

#32

Post by WhitneyLand »

dagmx wrote: post #17
amelius wrote: post #15

> Accelerated GPU training is enabled using Apple’s Metal Performance Shaders (MPS) as a backend for PyTorch. What do shaders have to do with it? Deep learning is a mature field now, it shouldn't need to borrow compute architecture from the gaming/entertainment field. Anyone else find this disconcerting?

Shaders are just the way compute is defined on the GPU. Why is that concerning to you?

It’s not the greatest term even for graphics only.

People new to CG are likely to intuit “shaders” as something related to, well, shading, but vertex shaders et al have nothing to do with the color of a pixel or a polygon.

Re: Accelerated PyTorch Training on M1 Mac

#33

Post by londons_explore »

LeanderK wrote: post #31
my123 wrote: post #11

Earlier quoted context omitted.

Low. Apple doesn't have matrix math accelerators in their current GPUs. The neural engine is small and inference only. It's also only exposed by a far higher level interface, CoreML. Where it could still make sense is if you have a small VRAM pool on the dGPU and a big one on the M1, but with the price of a Mac, not sure that makes a lot of sense either in most scenarios compared to paying for a big dGPU.

> The neural engine is small and inference only Why is it inference only? At least the operations are the same...just a bunch of linear algebra

Inference is often done fixed point, whereas training is (usually) floating point.

Inference also prefers different IO patterns, because you don't need to keep the activations for every layer ready for backpropogation.

Re: Accelerated PyTorch Training on M1 Mac

#35

Post by brrrrrm »

my123 wrote: post #23
Kon-Peki wrote: post #22

Earlier quoted context omitted.

> Apple doesn't have matrix math accelerators in their current GPUs. That's because the M1 has a dedicated matrix math accelerator called AMX [1]. I've used it with both Swift and pure C. https://medium.com/swlh/apples-m1-secret-coprocessor-6599492...

AMX is indeed very nice for FP64 where customer GPUs aren't an alternative at all. However, for lower precisions (which is what deep learning uses), you're much better off with a GPU.

have you actually benchmarked that? I think (someone please correct me if I'm way off here) the AMX instructions can hit ~2.8tflops (fp16) per co-processor and there are 2 on the 7-core M1. That's 5.6tflops vs the 4.6tflops the GPU can hit.

Re: Accelerated PyTorch Training on M1 Mac

#37

Post by galoisscobi »

alexfromapex wrote: post #34

Since it's tangentially relevant, if you have an M1 Mac I've created some boilerplate for working with the latest Tensorflow with GPU acceleration as well: https://github.com/alexfromapex/tensorexperiments . I'm thinking of adding a branch for PyTorch now.

This is great! Appreciate the note on H5Py troubleshooting as well.

Re: Accelerated PyTorch Training on M1 Mac

#39

Post by masklinn »

alexfromapex wrote: post #34

Since it's tangentially relevant, if you have an M1 Mac I've created some boilerplate for working with the latest Tensorflow with GPU acceleration as well: https://github.com/alexfromapex/tensorexperiments . I'm thinking of adding a branch for PyTorch now.

Did you compare that to Apple's tf plugin to see what was what?
Post reply on HN