Apple M1 support for TensorFlow 2.5 pluggable device API • Hacker News Board

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#21

Post by teruakohatu » Wed, Jun 09, 2021, 1:46 AM UTC

542458 wrote: post #2

Very cool. I’m not an ML guy, so apologies if this is a dumb question: how good is performance compared to other hardware here?

It is much, much slower than a dedicated GPU or TPU, so it is useful for "edge" inference eg. local speech or image recognition, but not for training new models. It is using shared memory with the CPU so I don't know how much memory you could eat up while training.

In short, you will probably get much better performance using Google Colab to train a deep learning model, than the M1.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#22

Post by xnyan » Wed, Jun 09, 2021, 1:47 AM UTC

fastball wrote: post #12

qmmmur wrote: post #8

Earlier quoted context omitted.

I run everything python under Rosetta. Easiest way is to install home brew for Intel processors then make an alias to that version for managing rosetta stuff

Kinda silly to use Rosetta when the whole point of this is native support for TensorFlow on M1.

Maybe, but lots and lots of software applications are not even close to being hardware constrained or in this specific case Rosetta constrained. If you're trying to get work done and you have the computational overhead available, this absolutely sounds like the right solution. Even if it's less efficient, I can easily imagine situations where "I need this to work without thinking about it" could win out.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#23

Post by codelord » Wed, Jun 09, 2021, 1:51 AM UTC

jra101 wrote: post #18

codelord wrote: post #5

M1 and AMD GPU support. I'm personally more interested in the latter as I haven't yet upgraded my MacBook Pro and I expect that my Vega 20 to be faster than M1 at ML training. The raw compute power of M1's GPU seems to be 2.6 TFLOPS (single precision) vs 3.2 TFLOPS for Vega 20. This can give you an estimate of how fast it would be for training. Just for reference Nvidia's flagship desktop GPU(3090)'s FP32 performance…

Vega 20 should be ~13.8 TFLOPs (single precision): https://www.anandtech.com/show/13923/the-amd-radeon-vii-revi...

Seems like AMD has been using Vega 20 to refer to two different things.

I was talking about the mobile GPU on MacBook Pros which is based on a 14nm chip. The full name is Radeon Pro Vega 20:

https://www.amd.com/en/graphics/radeon-pro-vega-20-pro-vega-...

https://www.techpowerup.com/gpu-specs/radeon-pro-vega-20.c32...

Vega 20 seems to also refer to a discrete GPU. This has been later rebranded to Radeon VII (maybe because of this confusion). The number you are quoting is for the discrete GPU.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#24

Post by notatoad » Wed, Jun 09, 2021, 1:56 AM UTC

rozim wrote: post #7

I have found the M1 air fine for web browsing but kind of hard to install software on. Following the instructions: ----- python -m pip install tensorflow-macos ... ERROR: Failed building wheel for numpy Failed to build numpy ERROR: Could not build wheels for numpy which use PEP 517 and cannot be installed directly ----- (base) dave@daves-air ~ % uname -a Darwin daves-air.lan 20.5.0 Darwin Kernel Version 20.5.0: Sat M…

`pip install --upgrade pip` fixed this for me. (not in tensorflow directly, but while installing something else on my M1 last week which required numpy)

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#25

Post by 6gvONxR4sf7o » Wed, Jun 09, 2021, 2:00 AM UTC

codelord wrote: post #5

M1 and AMD GPU support. I'm personally more interested in the latter as I haven't yet upgraded my MacBook Pro and I expect that my Vega 20 to be faster than M1 at ML training. The raw compute power of M1's GPU seems to be 2.6 TFLOPS (single precision) vs 3.2 TFLOPS for Vega 20. This can give you an estimate of how fast it would be for training. Just for reference Nvidia's flagship desktop GPU(3090)'s FP32 performance…

> Just for reference Nvidia's flagship GPU(3090)'s FP32 performance is 35.5 TFLOPS.

In context of ML, nvidia’s flagship is the A100, which has 312 TFLOPS. You can also compare with a TPU device which has 180 TFLOPS (v2) or 420 TFLOPS (v3). You can use at least the TPU v2 reliably on colab for free.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#26

Post by orik » Wed, Jun 09, 2021, 2:06 AM UTC

codelord wrote: post #5

M1 and AMD GPU support. I'm personally more interested in the latter as I haven't yet upgraded my MacBook Pro and I expect that my Vega 20 to be faster than M1 at ML training. The raw compute power of M1's GPU seems to be 2.6 TFLOPS (single precision) vs 3.2 TFLOPS for Vega 20. This can give you an estimate of how fast it would be for training. Just for reference Nvidia's flagship desktop GPU(3090)'s FP32 performance…

I wanted this for a long time, but eventually gave up on my MacPro5,1 that I had a Radeon VII in (before that, Vega Frontier, before that R9-280X).

Would like to see some benchmarks with that sort of hardware or with a 6900XT, driver support which came to MacOS only recently. Now I just have an M1 Mac Mini & a PC with Nvidia.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#27

Post by fartcannon » Wed, Jun 09, 2021, 2:07 AM UTC

6gvONxR4sf7o wrote: post #25

codelord wrote: post #5

M1 and AMD GPU support. I'm personally more interested in the latter as I haven't yet upgraded my MacBook Pro and I expect that my Vega 20 to be faster than M1 at ML training. The raw compute power of M1's GPU seems to be 2.6 TFLOPS (single precision) vs 3.2 TFLOPS for Vega 20. This can give you an estimate of how fast it would be for training. Just for reference Nvidia's flagship desktop GPU(3090)'s FP32 performance…

> Just for reference Nvidia's flagship GPU(3090)'s FP32 performance is 35.5 TFLOPS. In context of ML, nvidia’s flagship is the A100, which has 312 TFLOPS. You can also compare with a TPU device which has 180 TFLOPS (v2) or 420 TFLOPS (v3). You can use at least the TPU v2 reliably on colab for free.

The free part is limited to a set number of hours per month.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#28

Post by dragonwriter » Wed, Jun 09, 2021, 2:07 AM UTC

notatoad wrote: post #24

rozim wrote: post #7

I have found the M1 air fine for web browsing but kind of hard to install software on. Following the instructions: ----- python -m pip install tensorflow-macos ... ERROR: Failed building wheel for numpy Failed to build numpy ERROR: Could not build wheels for numpy which use PEP 517 and cannot be installed directly ----- (base) dave@daves-air ~ % uname -a Darwin daves-air.lan 20.5.0 Darwin Kernel Version 20.5.0: Sat M…

`pip install --upgrade pip` fixed this for me. (not in tensorflow directly, but while installing something else on my M1 last week which required numpy)

Presumably actually:

  python -m pip install --upgrade pip

just:

  pip install --upgrade pip

Is “break pip if there is a newer version available”, at least last I checked. (It uninstalls the existing pip but can’t complete the install of the new one.)

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#29

Post by NmAmDa » Wed, Jun 09, 2021, 2:11 AM UTC

dandiep wrote: post #9

I was able to install this fairly easily (much more so then the crap they dumped out here - https://github.com/apple/tensorflow_macos . Just take a look at the 200 github issues that were ignored for the most part...) I also noticed that in my project I got a decent speedup immediately when executing my model, but I have not run any benchmarks. But, where do you go to file bugs? Ask questions? etc. I am not a big Mac…

I think they don't track bugs there. They even archived the repository.

Re: Apple M1 support for TensorFlow 2.5 pluggable device API

#30

Post by codelord » Wed, Jun 09, 2021, 2:12 AM UTC

6gvONxR4sf7o wrote: post #25

codelord wrote: post #5

M1 and AMD GPU support. I'm personally more interested in the latter as I haven't yet upgraded my MacBook Pro and I expect that my Vega 20 to be faster than M1 at ML training. The raw compute power of M1's GPU seems to be 2.6 TFLOPS (single precision) vs 3.2 TFLOPS for Vega 20. This can give you an estimate of how fast it would be for training. Just for reference Nvidia's flagship desktop GPU(3090)'s FP32 performance…

> Just for reference Nvidia's flagship GPU(3090)'s FP32 performance is 35.5 TFLOPS. In context of ML, nvidia’s flagship is the A100, which has 312 TFLOPS. You can also compare with a TPU device which has 180 TFLOPS (v2) or 420 TFLOPS (v3). You can use at least the TPU v2 reliably on colab for free.

It's not really fair to compare a discrete GPU to a mobile GPU, I only provided this as a comparison for someone who maybe has one of these at home. And btw, you are talking about TF32 performance not FP32. TF32 actually uses 16 bits. A100's FP32 performance is actually lower than 3090, it's 19.5 TFLOPS: https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Cent...

FP16 performance is also relevant as a lot of people now train in FP16. The default for pytorch/TF is still FP32.