Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

wandb.ai

Post reply on HN
Print view View original

71–80 of 94 posts

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#71

Post by lukas »

volta87 wrote: post #49

When developing ML models, you rarely train "just one". The article mentions that they explored a not-so-large hyper-parameter space (i.e. they trained multiple models with different parameters each). It would be interesting to know how long does the whole process takes on the M1 vs the V100. For the small models covered in the article, I'd guess that the V100 can train them all concurrently using MPS (multi-process…

Do you really train more than one model at the same time on a single GPU? In my experience that's pretty unusual.

I completely agree with your conclusion here.

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#72

Post by lukas »

tbalsam wrote: post #70

This is on a model designed to run faster on CPUs. It's like dropping a bowling ball on your foot and claiming excitement that you feel bruised after a few days. Maybe there's something interesting there, definitely, but the overhype of the title takes away any significant amount of clout I'd give to the publishers for research. If you find something interesting, say it, and stop making vapid generalizations for the…

I don't think MobileNetV2 is designed to train on GPUs - according to this https://azure.microsoft.com/en-us/blog/gpus-vs-cpus-for-depl... MobileNetV2 gets bigger gains from GPUs vs several CPUs than ResNet. You could argue the batch size doesn't fully use the V100 but these comparisons are tricky and this looks like fairly normal training to me.

It's pretty surprising to me that an M1 performs anywhere near a V100 on model training and I guess the most striking thing is the energy efficiency of the M1.

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#73

Post by tbalsam »

lukas wrote: post #72
tbalsam wrote: post #70

This is on a model designed to run faster on CPUs. It's like dropping a bowling ball on your foot and claiming excitement that you feel bruised after a few days. Maybe there's something interesting there, definitely, but the overhype of the title takes away any significant amount of clout I'd give to the publishers for research. If you find something interesting, say it, and stop making vapid generalizations for the…

I don't think MobileNetV2 is designed to train on GPUs - according to this https://azure.microsoft.com/en-us/blog/gpus-vs-cpus-for-depl... MobileNetV2 gets bigger gains from GPUs vs several CPUs than ResNet. You could argue the batch size doesn't fully use the V100 but these comparisons are tricky and this looks like fairly normal training to me. It's pretty surprising to me that an M1 performs anywhere near a V100 o…

MV2 is memory-limited, the depthwise + groups + 1x1 convs has a long launch time on GPU. Shattered kernels are fine for CPU, but not for GPU.

Though per your note on the scales, that's really interesting empirical results. I'll have to look into that, thanks for passing that along.

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#74

Post by sdenton4 »

volta87 wrote: post #49

When developing ML models, you rarely train "just one". The article mentions that they explored a not-so-large hyper-parameter space (i.e. they trained multiple models with different parameters each). It would be interesting to know how long does the whole process takes on the M1 vs the V100. For the small models covered in the article, I'd guess that the V100 can train them all concurrently using MPS (multi-process…

The low gpu utilization rate in the first graph is kind of a tell... Seems like the M1 is a little bit worse than 40% of a v100?

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#75

Post by [deleted] »

tbalsam wrote: post #70

This is on a model designed to run faster on CPUs. It's like dropping a bowling ball on your foot and claiming excitement that you feel bruised after a few days. Maybe there's something interesting there, definitely, but the overhype of the title takes away any significant amount of clout I'd give to the publishers for research. If you find something interesting, say it, and stop making vapid generalizations for the…

[deleted]

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#76

Post by vanpelt »

SloopJon wrote: post #64

The first graph includes "Apple Intel", which is not mentioned anywhere else in the post. Any idea what hardware that was, and whether it used the accelerated TensorFlow?

My bad, this was using non-Accelerated TensorFlow on a 2.3GHz 8-Core i9.

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#78

Post by iluxonchik »

mark_l_watson wrote: post #51

I had the same experience. My M1 system does well on smaller models compared to a NVidia 1070 with 10GB of memory. My MacBook Pro only has 8GB total memory. Large models run slowly. I found setting up Apple’s M1 fork of TensorFlow to be fairly easy, BTW. I am writing a new book on using Swift for AI applications, motivated by the “niceness” of the Swift language and Apple’s CoreML libraries.

do you happen to have a draft version available somewhere? i'm diving into ML with Swift soon

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#79

Post by ZeroCool2u »

StavrosK wrote: post #77

I'm seeing a lot of M1 hype, and I suspect most of it us unwarranted. I looked at comparisons between the M1 and the latest Ryzens, and it looks like it's comparable? Does anyone know details? I only looked summarily.

The main hype is that performance is similar, but the M1 does it with a lot less power draw. The performance itself isn't too crazy. It's just crazy that it does it with a somewhat similar power draw to a high end phone.

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#80

Post by junipertea »

lukas wrote: post #71
volta87 wrote: post #49

When developing ML models, you rarely train "just one". The article mentions that they explored a not-so-large hyper-parameter space (i.e. they trained multiple models with different parameters each). It would be interesting to know how long does the whole process takes on the M1 vs the V100. For the small models covered in the article, I'd guess that the V100 can train them all concurrently using MPS (multi-process…

Do you really train more than one model at the same time on a single GPU? In my experience that's pretty unusual. I completely agree with your conclusion here.

I found training multiple models on same GPU hit other bottlenecks (mainly memory capacity/bandwidth) fast. I tend to train one model per GPU and just scale the number of computers. Also, if nothing else, we tend to push the models to fit the GPU memory.
Post reply on HN