Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accelerated PyTorch Training on M1 Mac

pytorch.org

Post reply on HN
Print view View original

111–120 of 153 posts

Re: Accelerated PyTorch Training on M1 Mac

#111

Post by dekhn »

iamaaditya wrote: post #57
sillyinseattle wrote: post #49

Earlier quoted context omitted.

Question about terminology (no background in AI). In econometrics, estimation is model fitting (training, I guess), and inference refers to hypothesis testing (e.g. t or F tests). What does inference mean here?

In machine learning (especially deep learning or neural networks), the 'training' is done by using Stochastic Gradient Descent. These gradients are computed using Backpropagation. Backpropagation requires you to do a backward pass of your model (typically many layers of neural weights) and thus requires you to keep in memory a lot of intermediate values (called activations). However, if you are doing "inference" that…

it took me 20 years to learn this body of knowledge and now it can just sort of be summed up in a paragraph.

When I learned and used gradient descent, you had to analytically determine your own gradients (https://web.archive.org/web/20161028022707/https://genomics....). I went to grad school to learn how to determine my own gradients. Unfortunately, in my realm, loss landscapes have multiple minima, and gradient descent just gets trapped in local minima.

Re: Accelerated PyTorch Training on M1 Mac

#114

Post by jacobn »

in3d wrote: post #112

It’s surprising to see PyTorch developers working on things like that when common operations like group convolutions are still completely unoptimized on Nvidia GPUs, despite many requests.

Grouped convolutions can't really run faster than groups * conv(ch/group) and I believe that's close to where they're at?

Note that for chSo unfortunately depthwise convolutions end up having terrible performance.

Re: Accelerated PyTorch Training on M1 Mac

#115

Post by thentherewere2 »

dekhn wrote: post #111
iamaaditya wrote: post #57

Earlier quoted context omitted.

In machine learning (especially deep learning or neural networks), the 'training' is done by using Stochastic Gradient Descent. These gradients are computed using Backpropagation. Backpropagation requires you to do a backward pass of your model (typically many layers of neural weights) and thus requires you to keep in memory a lot of intermediate values (called activations). However, if you are doing "inference" that…

it took me 20 years to learn this body of knowledge and now it can just sort of be summed up in a paragraph. When I learned and used gradient descent, you had to analytically determine your own gradients ( https://web.archive.org/web/20161028022707/https://genomics.... ). I went to grad school to learn how to determine my own gradients. Unfortunately, in my realm, loss landscapes have multiple minima, and gradient de…

This is the case most contemporary neural networks as well. It turns out for many domains, a "good" local minima generalizes well across many tasks.

Re: Accelerated PyTorch Training on M1 Mac

#116

Post by dekhn »

thentherewere2 wrote: post #115
dekhn wrote: post #111

Earlier quoted context omitted.

it took me 20 years to learn this body of knowledge and now it can just sort of be summed up in a paragraph. When I learned and used gradient descent, you had to analytically determine your own gradients ( https://web.archive.org/web/20161028022707/https://genomics.... ). I went to grad school to learn how to determine my own gradients. Unfortunately, in my realm, loss landscapes have multiple minima, and gradient de…

This is the case most contemporary neural networks as well. It turns out for many domains, a "good" local minima generalizes well across many tasks.

Huh. I talked to some experts and they told me NN loss functions are bowl-shaped and have single minima, but those minima take a very long time to navigate to in high dimensional spaces.

Re: Accelerated PyTorch Training on M1 Mac

#117

Post by in3d »

jacobn wrote: post #114
in3d wrote: post #112

It’s surprising to see PyTorch developers working on things like that when common operations like group convolutions are still completely unoptimized on Nvidia GPUs, despite many requests.

Grouped convolutions can't really run faster than groups * conv(ch/group) and I believe that's close to where they're at? Note that for ch So unfortunately depthwise convolutions end up having terrible performance.

Why wouldn’t you be able to run them in parallel using CUDA? You shouldn’t be memory-transfer speed limited when group convolution layers are a part of a bigger net.

Note that pointwise 1x1 convolutions are a special case of group convolutions and actually I think they might be specially optimized in PyTorch (I’d have to run some benchmarks to test it though).

Re: Accelerated PyTorch Training on M1 Mac

#118

Post by blihp »

ribit wrote: post #110
ActorNightly wrote: post #68

Earlier quoted context omitted.

> but they're already way ahead on energy efficiency. For raw compute like you need for ML training, the M1s efficiency doesn't matter. Under the hood at hardware level, you have a direct mapping of power consumption to compute circuit activation that you really can't get around. The general efficiency of M1 is due its architecture and how it fits together with normal consumer use. Less stuff on the instruction decod…

And yet somehow Apples GPU ALUs are more efficient at 3.8 watts per TFLOP. Mind, I am not talking about specialized matrix multiplication units that have a different internal organization and can do things like matrix multiplication much more efficiently, but about basic general-purpose GPU ALUs. The comparison of efficiency between Apple and Nvidia here is a bit misleading because one compares Apples general-purpose…

How they do it at a conceptual level isn't a big secret: they don't need to minimize die area the way other companies do. For Apple, the die is just part of a chip that is part of the larger system they sell that they can amortize the cost over. nVidia doesn't have a system to do that with so their natural inclination is to lean towards keeping the die size as small as possible and just overclock the hell out of it. (right there is the 'trick': Apple can afford to do things that chew up die space that nVidia and others can't while maintaining their profit margins) Being a process generation ahead is also a rather huge thing too. (which is another thing they can amortize the cost of over large numbers of complete systems and mobile devices which their competitors can't)

Also related: Apple designs their hardware to do just what they want it to while everyone else is designing for a more general use case. This also costs die area, IP licensing fees etc.

Re: Accelerated PyTorch Training on M1 Mac

#119

Post by ekelsen »

jorgemf wrote: post #30
ekelsen wrote: post #10

Nice results! But why are people still reporting benchmark results on VGG? Does anybody actually use this network anymore? Better would be mobilenets or efficientNets or NFNets or vision transformers or almost anything that's come out in the 8 years since VGG was published (great work it was at the time!).

Probably because it will be impossible to compare with old results. If every year the community chooses a different model, how are you going to compare results year over year?

The numbers are relative speedups, not absolute numbers that can be compared with any prior results, so I don't really see how this matters.

Re: Accelerated PyTorch Training on M1 Mac

#120

Post by Salgat »

dekhn wrote: post #116
thentherewere2 wrote: post #115

Earlier quoted context omitted.

This is the case most contemporary neural networks as well. It turns out for many domains, a "good" local minima generalizes well across many tasks.

Huh. I talked to some experts and they told me NN loss functions are bowl-shaped and have single minima, but those minima take a very long time to navigate to in high dimensional spaces.

For higher feature counts the real concern is saddle points rather than minima, where the gradient is so small that you barely move at all each iteration and get "stuck".
Post reply on HN