Does speed up refer to absolute value or percentage?
At least for the charts, it looks like a multiplier (or divisor I guess) since the CPU baseline looks to be at 1
Accelerated PyTorch Training on M1 Mac
21–30 of 153 posts
Re: Accelerated PyTorch Training on M1 Mac
#22I'm curious about the performance compared to something like, say, the RTX 3070.
Low. Apple doesn't have matrix math accelerators in their current GPUs. The neural engine is small and inference only. It's also only exposed by a far higher level interface, CoreML. Where it could still make sense is if you have a small VRAM pool on the dGPU and a big one on the M1, but with the price of a Mac, not sure that makes a lot of sense either in most scenarios compared to paying for a big dGPU.
> Apple doesn't have matrix math accelerators in their current GPUs.
That's because the M1 has a dedicated matrix math accelerator called AMX [1]. I've used it with both Swift and pure C.
https://medium.com/swlh/apples-m1-secret-coprocessor-6599492...
Re: Accelerated PyTorch Training on M1 Mac
#23Earlier quoted context omitted.
Low. Apple doesn't have matrix math accelerators in their current GPUs. The neural engine is small and inference only. It's also only exposed by a far higher level interface, CoreML. Where it could still make sense is if you have a small VRAM pool on the dGPU and a big one on the M1, but with the price of a Mac, not sure that makes a lot of sense either in most scenarios compared to paying for a big dGPU.
> Apple doesn't have matrix math accelerators in their current GPUs. That's because the M1 has a dedicated matrix math accelerator called AMX [1]. I've used it with both Swift and pure C. https://medium.com/swlh/apples-m1-secret-coprocessor-6599492...
AMX is indeed very nice for FP64 where customer GPUs aren't an alternative at all.
However, for lower precisions (which is what deep learning uses), you're much better off with a GPU.
Re: Accelerated PyTorch Training on M1 Mac
#24Nice results! But why are people still reporting benchmark results on VGG? Does anybody actually use this network anymore? Better would be mobilenets or efficientNets or NFNets or vision transformers or almost anything that's come out in the 8 years since VGG was published (great work it was at the time!).
> But why are people still reporting benchmark results on VGG?
It makes me feel like i’m missing something! Is is still used as a backbone in the same way as legacy code is everywhere, or is it something else entirely??
Re: Accelerated PyTorch Training on M1 Mac
#25Earlier quoted context omitted.
Shaders are just the way compute is defined on the GPU. Why is that concerning to you?
That terminology isn't used at all in GPGPU compute APIs specifically tailored for that purpose, which use quite different programming models where you can mix host and device code in the same program. And there are "GPUs" today that can't do graphics at all (AMD MI100/MI200 generations) or in a restricted way (Hopper GH100) which has the fixed function pipeline only on two TPCs, for compatibility, but running very s…
There's absolutely a lot of "graphics" terminology that spills into GPGPU. For example, texture memory in CUDA :) The reality is that GPU's, even the ones that can't output video, are ultimately still using hardware that largely is rooted in gaming. Obviously the underlying architectures for these ML cards are moving away from that (increasingly using more die space for ML related operations) but many of the core components like memory are still shared. It boils down to the fact that at the end of the day they're linear algebra processors.
Re: Accelerated PyTorch Training on M1 Mac
#26Re: Accelerated PyTorch Training on M1 Mac
#27Nice results! But why are people still reporting benchmark results on VGG? Does anybody actually use this network anymore? Better would be mobilenets or efficientNets or NFNets or vision transformers or almost anything that's come out in the 8 years since VGG was published (great work it was at the time!).
> Does anybody actually use this network anymore?
Why not? It's still good for simple classification tasks. We use it as an encoder for a segmentation model in some cases. Most ResNet variants are much heavier.
Re: Accelerated PyTorch Training on M1 Mac
#28Earlier quoted context omitted.
That terminology isn't used at all in GPGPU compute APIs specifically tailored for that purpose, which use quite different programming models where you can mix host and device code in the same program. And there are "GPUs" today that can't do graphics at all (AMD MI100/MI200 generations) or in a restricted way (Hopper GH100) which has the fixed function pipeline only on two TPCs, for compatibility, but running very s…
There's absolutely a lot of "graphics" terminology that spills into GPGPU. For example, texture memory in CUDA :) The reality is that GPU's, even the ones that can't output video, are ultimately still using hardware that largely is rooted in gaming. Obviously the underlying architectures for these ML cards are moving away from that (increasingly using more die space for ML related operations) but many of the core com…
I'd say that there has been quite some sharing between both back and forth. Evolutions in compute stacks shaped modern graphics APIs too.
Texture units are indeed a part that is useful enough to be exposed to GPGPU compute APIs directly. The "shader" term itself disappeared quite early in those though, as did access to a good part of the FF pipeline including the rasterisers themselves.
Re: Accelerated PyTorch Training on M1 Mac
#29Nice results! But why are people still reporting benchmark results on VGG? Does anybody actually use this network anymore? Better would be mobilenets or efficientNets or NFNets or vision transformers or almost anything that's come out in the 8 years since VGG was published (great work it was at the time!).
why are people still reporting benchmark results on VGG?
Probably because it makes the hardware look good.
Re: Accelerated PyTorch Training on M1 Mac
#30Nice results! But why are people still reporting benchmark results on VGG? Does anybody actually use this network anymore? Better would be mobilenets or efficientNets or NFNets or vision transformers or almost anything that's come out in the 8 years since VGG was published (great work it was at the time!).
Probably because it will be impossible to compare with old results. If every year the community chooses a different model, how are you going to compare results year over year?