Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accelerated PyTorch Training on M1 Mac

pytorch.org

Post reply on HN
Print view View original

41–50 of 153 posts

Re: Accelerated PyTorch Training on M1 Mac

#41

Post by 0-_-0 »

p1esk wrote: post #29
ekelsen wrote: post #10

Nice results! But why are people still reporting benchmark results on VGG? Does anybody actually use this network anymore? Better would be mobilenets or efficientNets or NFNets or vision transformers or almost anything that's come out in the 8 years since VGG was published (great work it was at the time!).

why are people still reporting benchmark results on VGG? Probably because it makes the hardware look good.

This is the right answer. Efficient networks like EfficientNet are much harder to accelerate in HW.

Re: Accelerated PyTorch Training on M1 Mac

#44

Post by johndough »

brrrrrm wrote: post #35
my123 wrote: post #23

Earlier quoted context omitted.

AMX is indeed very nice for FP64 where customer GPUs aren't an alternative at all. However, for lower precisions (which is what deep learning uses), you're much better off with a GPU.

have you actually benchmarked that? I think (someone please correct me if I'm way off here) the AMX instructions can hit ~2.8tflops (fp16) per co-processor and there are 2 on the 7-core M1. That's 5.6tflops vs the 4.6tflops the GPU can hit.

Often the limiting factor is memory bandwidth instead of raw FLOPS, so dealing with 4 times larger data types (FP64 vs FP16) is a disadvantage.

Re: Accelerated PyTorch Training on M1 Mac

#45

Post by sudosysgen »

my123 wrote: post #16
amelius wrote: post #15

> Accelerated GPU training is enabled using Apple’s Metal Performance Shaders (MPS) as a backend for PyTorch. What do shaders have to do with it? Deep learning is a mature field now, it shouldn't need to borrow compute architecture from the gaming/entertainment field. Anyone else find this disconcerting?

Apple doesn't have a separate API tailored towards compute only, but a single unified API that makes concessions to both. Concessions towards compute: a C++ programming language for device code (totally unlike what's done for most graphics APIs!) Concessions towards graphics: no single-source programming model at all for example...

Many GPUs allow you to write device code in C++ via SYCL. It works well enough.

Re: Accelerated PyTorch Training on M1 Mac

#47

Post by brrrrrm »

johndough wrote: post #44
brrrrrm wrote: post #35

Earlier quoted context omitted.

have you actually benchmarked that? I think (someone please correct me if I'm way off here) the AMX instructions can hit ~2.8tflops (fp16) per co-processor and there are 2 on the 7-core M1. That's 5.6tflops vs the 4.6tflops the GPU can hit.

Often the limiting factor is memory bandwidth instead of raw FLOPS, so dealing with 4 times larger data types (FP64 vs FP16) is a disadvantage.

to clarify: I am comparing FP16 performance, which both the GPU and AMX have native support for.

FP64 is also supported by AMX, making it quite an impressive region of silicon.

Re: Accelerated PyTorch Training on M1 Mac

#48

Post by mkaic »

This is really cool for a number of reasons:

1.) Apple Silicon currently can't compete with Nvidia GPUs in terms of raw compute power, but they're already way ahead on energy efficiency. Training a small deep learning model on battery power on a laptop could actually be a thing now.

Edit: I've been informed that for matrix math, Apple Silicon isn't actually ahead in efficiency

2.) Apple Silicon probably will compete directly with Nvidia GPUs in the near future in terms of raw compute power in future generations of products like the Mac Studio and Mac Pro, which is very exciting. Competition in this space is incredibly good for consumers.

3.) At $4800, an M1 Ultra Mac Studio appears to be far and away the cheapest machine you can buy with 128GB of GPU memory. With proper PyTorch support, we'll actually be able to use this memory for training big models or using big batch sizes. For the kind of DL work I do where dataloading is much more of a bottleneck than actual raw compute power, Mac Studio is now looking very enticing.

Re: Accelerated PyTorch Training on M1 Mac

#49

Post by sillyinseattle »

RicoElectrico wrote: post #42
lekevicius wrote: post #36

Curiously neither PyTorch nor Tensorflow currently use M1's Neural Engine. Is too limited? Too hard to interact with? Not worth the effort?

Most probably Neural Engine is optimized for inference, not training.

Question about terminology (no background in AI). In econometrics, estimation is model fitting (training, I guess), and inference refers to hypothesis testing (e.g. t or F tests). What does inference mean here?
Post reply on HN