Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

wandb.ai

Post reply on HN
Print view View original

81–90 of 94 posts

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#81

Post by littlestymaar »

rrss wrote: post #68
littlestymaar wrote: post #19

Earlier quoted context omitted.

> The special thing about the V100 is that it's driver EULA allows data center usage. Wait what? Is it the only thing? That sounds hard to believe: if true, using the open driver (Nouveau) instead of Nvidia's proprietary one would be a massive money saver for datacenters operators (and even if Nouveau doesn't support the features you'd want already, supporting their development would be much cheaper for a company lik…

No, that's not the only thing. Other characteristics of V100 that may be interesting to people buying GPUs for data centers: - higher capacity GPU memory. 1080 has 8 GB, V100 has 16 or 32 GB. - higher bandwidth GPU memory. V100 has HBM2 with a peak of 900 GB/s, 1080 has G5X with a peak of ~300 GB/s. - ECC support. - data center certification + warranty (The geforce warranty covers normal consumer usage, like gaming,…

Thanks, that makes much more sense!

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#82

Post by sdenton4 »

sillysaurusx wrote: post #18
lopuhin wrote: post #6

> I chose MobileNetV2 to make iteration faster. When I tried ResNet50 or other larger models the gap between the M1 and Nvidia grew wider. (and that's on CIFAR-10). But why not report these results and also test on a more realistic datasets? The internet is full of M1 TF brenchmarks on CIFAR or MNIST, has anyone seen something different?

Hehe. That criticism could be applied to ML itself. :) I wish ML used more than CIFNISTNet, but unfortunately there's not a lot of standard datasets yet. (Even Imagenet is an absolute pain to set up.)

Tensorflow Datasets includes a lot of the 'standard' datasets in a way that's dead simple to call up and use (including ~10 variants of imagenet): https://www.tensorflow.org/datasets/catalog/overview

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#83

Post by figomore »

iluxonchik wrote: post #78
mark_l_watson wrote: post #51

I had the same experience. My M1 system does well on smaller models compared to a NVidia 1070 with 10GB of memory. My MacBook Pro only has 8GB total memory. Large models run slowly. I found setting up Apple’s M1 fork of TensorFlow to be fairly easy, BTW. I am writing a new book on using Swift for AI applications, motivated by the “niceness” of the Swift language and Apple’s CoreML libraries.

do you happen to have a draft version available somewhere? i'm diving into ML with Swift soon

I think this is the book https://leanpub.com/SwiftAI

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#85

Post by Firadeoclus »

rrss wrote: post #68
littlestymaar wrote: post #19

Earlier quoted context omitted.

> The special thing about the V100 is that it's driver EULA allows data center usage. Wait what? Is it the only thing? That sounds hard to believe: if true, using the open driver (Nouveau) instead of Nvidia's proprietary one would be a massive money saver for datacenters operators (and even if Nouveau doesn't support the features you'd want already, supporting their development would be much cheaper for a company lik…

No, that's not the only thing. Other characteristics of V100 that may be interesting to people buying GPUs for data centers: - higher capacity GPU memory. 1080 has 8 GB, V100 has 16 or 32 GB. - higher bandwidth GPU memory. V100 has HBM2 with a peak of 900 GB/s, 1080 has G5X with a peak of ~300 GB/s. - ECC support. - data center certification + warranty (The geforce warranty covers normal consumer usage, like gaming,…

A GTX1080 manages about ~9 TFLOPS(fp32) (and has terrible fp16 support), where V100 gets ~15 TFLOPS(fp16), ~30 TFLOPS(fp16), and ~120 TFLOPS(tensor cores).

Apart from one being a gaming product and the other being designed for computational tasks, they're a generation apart and have various small differences that may be quite relevant for individual tasks (such as V100 allowing twice the shared memory - 96 KiB - per thread block)

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#86

Post by alickz »

jklehm wrote: post #58
alickz wrote: post #53

Earlier quoted context omitted.

They barely allow single buying for the 30 series :( Took me quite a while to get my hands on a 3080.

What ended up working for you?

I bought from a (relatively) small German commerce site[1] rather than a bigger site like Amazon, OCUK, or Scan. I'm in EU though, probably doesn't help if you're US. I think I paid a €50 or so premium over the retail price but I didn't mind that too much.

I used this[2] site to keep an eye open for stock, as you can see it's pretty much empty now but I just checked every day and finally found one.

[1] https://www.reichelt.de/ [2] https://www.gputracker.eu/en/search/category/1/graphics-card...

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#87

Post by jklehm »

alickz wrote: post #86
jklehm wrote: post #58

Earlier quoted context omitted.

What ended up working for you?

I bought from a (relatively) small German commerce site[1] rather than a bigger site like Amazon, OCUK, or Scan. I'm in EU though, probably doesn't help if you're US. I think I paid a €50 or so premium over the retail price but I didn't mind that too much. I used this[2] site to keep an eye open for stock, as you can see it's pretty much empty now but I just checked every day and finally found one. [1] https://www.re…

Thanks for the insights...frustrating times to be searching for one.

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#88

Post by mark_l_watson »

figomore wrote: post #83
iluxonchik wrote: post #78

Earlier quoted context omitted.

do you happen to have a draft version available somewhere? i'm diving into ML with Swift soon

I think this is the book https://leanpub.com/SwiftAI

Thanks for linking that, but there is not much of the book written for now. I have mostly been working on the examples.

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#89

Post by mark_l_watson »

iluxonchik wrote: post #78
mark_l_watson wrote: post #51

I had the same experience. My M1 system does well on smaller models compared to a NVidia 1070 with 10GB of memory. My MacBook Pro only has 8GB total memory. Large models run slowly. I found setting up Apple’s M1 fork of TensorFlow to be fairly easy, BTW. I am writing a new book on using Swift for AI applications, motivated by the “niceness” of the Swift language and Apple’s CoreML libraries.

do you happen to have a draft version available somewhere? i'm diving into ML with Swift soon

If you are interested in just the iOS/iPadOS/macOS platforms, then work through the tutorial articles on ML that Apple provides to devs.

If you are on Linux, then Swift for TensorFlow is OK. You will save some effort by using Google Colab notebooks, that support Swift and Swift for TensorFlow.

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#90

Post by volta87 »

lukas wrote: post #71
volta87 wrote: post #49

When developing ML models, you rarely train "just one". The article mentions that they explored a not-so-large hyper-parameter space (i.e. they trained multiple models with different parameters each). It would be interesting to know how long does the whole process takes on the M1 vs the V100. For the small models covered in the article, I'd guess that the V100 can train them all concurrently using MPS (multi-process…

Do you really train more than one model at the same time on a single GPU? In my experience that's pretty unusual. I completely agree with your conclusion here.

Depends on model size, but if the model is small enough that I actually do training on a PCIe board, I do. I partition an A100 in 8, and train 8 models at a time, or just use MPS on a V100 board. The bigger A100 boards can fit multiple of the same models that do fit in a single V100..

Also I tend to do this initially, when I am exploring the hyperparameter space, for which I tend to use smaller but more models.

I find that using big models initially is just a waste of time. You want to try many things as quickly as possible.

Post reply on HN