Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Accelerated PyTorch Training on M1 Mac

pytorch.org

Post reply on HN
Print view View original

131–140 of 153 posts

Re: Accelerated PyTorch Training on M1 Mac

#131

Post by in3d »

brrrrrm wrote: post #128
in3d wrote: post #117

Earlier quoted context omitted.

Why wouldn’t you be able to run them in parallel using CUDA? You shouldn’t be memory-transfer speed limited when group convolution layers are a part of a bigger net. Note that pointwise 1x1 convolutions are a special case of group convolutions and actually I think they might be specially optimized in PyTorch (I’d have to run some benchmarks to test it though).

pointwise isn’t a case of grouped conv, they’re orthogonal ideas. You can fuse grouped convs (depthwise is a special case of grouped convs) into preceding or following layers. Maybe JAX can do this already? No clue if any library offers such an optimization out of the box

Sorry, yes, I was replying to the post about depthwise convolution and that’s what I meant (though the naming of it is poor) - i.e. the special case of group convolutions where the number of groups is equal to the number of channels.

Re: Accelerated PyTorch Training on M1 Mac

#132

Post by fragmede »

almostdigital wrote: post #129

Anyone actually got this to run on an M1 Mac? $ conda install pytorch torchvision torchaudio -c pytorch-nightly Collecting package metadata (current_repodata.json): done Solving environment: failed with initial frozen solve. Retrying with flexible solve. Collecting package metadata (repodata.json): done Solving environment: failed with initial frozen solve. Retrying with flexible solve. PackagesNotFoundError: The fol…

    pip3 install pytorch
worked for me. I think it's something with your brew installation.

    fragmede@samairmac:~$ python
    Python 3.9.7 | packaged by conda-forge | (default, Sep 29 2021, 19:24:02)
    [Clang 11.1.0 ] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import torch
    >>> torch.__file__
    '/Users/fragmede/projects/miniforge3/lib/python3.9/sitepackages/torch/__init__.py'
    >>>

Re: Accelerated PyTorch Training on M1 Mac

#134

Post by eoerl »

p1esk wrote: post #73
sudosysgen wrote: post #66

Earlier quoted context omitted.

Apple Silicon is not ahead at all on energy efficiency for desktop workloads. If they were ahead on energy efficiency, they would simply be ahead on power. Indeed, GPUs are massively parallel architectures, and they are generally limited by the transistor and power budget (and memory, of course). Apple is simply behind in the GPU space. > At $4800, an M1 Ultra Mac Studio appears to be far and away the cheapest machin…

its memory is at a fraction (around 30-40%) of the memory bandwidth of a 128GB equivalent GPU setup Here's some info about M1 memory bandwidth: https://www.anandtech.com/show/17024/apple-m1-max-performanc...

I'm not sure what you meant with the link, but the parent is right, so adding an explanation here: M1 Ultra has about 400GB/s theoretical bandwidth but Anandtech shows that none of the SoC blocks can actually reach that, pretty far for it. It seems that Apple summed all the bandwidth to all the blocks to get there, which does mean something but not that the GPU has access to this (the GPU memory controllers seem to be the bottleneck).

On the contrary, a 3080 laptop does reach 400GB/s, I'm personally seeing this routinely on AI workloads, so that's part of the explanation for subpar perf here (the other ones being probably matrix math and mixed precision)

Re: Accelerated PyTorch Training on M1 Mac

#135

Post by eoerl »

ekelsen wrote: post #10

Nice results! But why are people still reporting benchmark results on VGG? Does anybody actually use this network anymore? Better would be mobilenets or efficientNets or NFNets or vision transformers or almost anything that's come out in the 8 years since VGG was published (great work it was at the time!).

actually it seems that it was because a lot of other well known models are not yet supported, missing ops in the Metal backend

Re: Accelerated PyTorch Training on M1 Mac

#136

Post by almostdigital »

fragmede wrote: post #132
almostdigital wrote: post #129

Anyone actually got this to run on an M1 Mac? $ conda install pytorch torchvision torchaudio -c pytorch-nightly Collecting package metadata (current_repodata.json): done Solving environment: failed with initial frozen solve. Retrying with flexible solve. Collecting package metadata (repodata.json): done Solving environment: failed with initial frozen solve. Retrying with flexible solve. PackagesNotFoundError: The fol…

pip3 install pytorch worked for me. I think it's something with your brew installation. fragmede@samairmac:~$ python Python 3.9.7 | packaged by conda-forge | (default, Sep 29 2021, 19:24:02) [Clang 11.1.0 ] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import torch >>> torch.__file__ '/Users/fragmede/projects/miniforge3/lib/python3.9/sitepackages/torch/__init__.py' >>>

Does torchaudio work for you? I can get torch and torchvision to work but not torchaudio

Re: Accelerated PyTorch Training on M1 Mac

#137

Post by andoma »

omegalulw wrote: post #99
jph00 wrote: post #85

Earlier quoted context omitted.

Using mixed precision training you can do most operations in fp16 and just a few in fp32 where it's needed. This is the norm for NVIDIA GPU training nowadays. For instance using fastai add `.to_fp16()` after your learner call, and that happens automatically.

How is the choice between fp16 and fp32 made? Is it like if any gradients in the tensor need the extra range you use fp32?

This article [0] from Nvidia gives a good overview of how mixed precision training works.

Super high level (from section 3):

  1. Converting the model to use the float16 data type where possible.
  2. Keeping float32 master weights to accumulate per-iteration weight updates.
  3. Using loss scaling to preserve small gradient values.
[0] https://docs.nvidia.com/deeplearning/performance/mixed-preci...

Re: Accelerated PyTorch Training on M1 Mac

#138

Post by timomo »

Salgat wrote: post #120
dekhn wrote: post #116

Earlier quoted context omitted.

Huh. I talked to some experts and they told me NN loss functions are bowl-shaped and have single minima, but those minima take a very long time to navigate to in high dimensional spaces.

For higher feature counts the real concern is saddle points rather than minima, where the gradient is so small that you barely move at all each iteration and get "stuck".

To add here: for a local minimum to occur all those dimensions (or features) need to increase. This is highly unlikely for modern NNs where you have millions of dimensions. If one of the dimensions is going down but the rest up, you have a saddle point. Since you go down only one (or few) dimensions it takes longer.

Re: Accelerated PyTorch Training on M1 Mac

#139

Post by KingOfCoders »

blihp wrote: post #118
ribit wrote: post #110

Earlier quoted context omitted.

And yet somehow Apples GPU ALUs are more efficient at 3.8 watts per TFLOP. Mind, I am not talking about specialized matrix multiplication units that have a different internal organization and can do things like matrix multiplication much more efficiently, but about basic general-purpose GPU ALUs. The comparison of efficiency between Apple and Nvidia here is a bit misleading because one compares Apples general-purpose…

How they do it at a conceptual level isn't a big secret: they don't need to minimize die area the way other companies do. For Apple, the die is just part of a chip that is part of the larger system they sell that they can amortize the cost over. nVidia doesn't have a system to do that with so their natural inclination is to lean towards keeping the die size as small as possible and just overclock the hell out of it.…

Also I thought Apple is adding a large slice of cache. When you look at the 3D-V cache on Ryzen for performance (+15%?), this has a large impact. And because they sell expensive stuff, they can afford to build expensive CPUs.

Re: Accelerated PyTorch Training on M1 Mac

#140

Post by nxpnsv »

MasterScrat wrote: post #2

Small code example in the PyTorch doc: https://pytorch.org/docs/master/notes/mps.html

Tried https://pytorch.org/tutorials/beginner/basics/quickstart_tut... with mps vs cpu. mps worked, but cpu actually was faster (16 vs 21s). Perhaps I am doing it wrong...
Post reply on HN