Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100 • Hacker News Board

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#91

Post by volta87 » Fri, Jan 15, 2021, 5:40 PM UTC

sdenton4 wrote: post #74

volta87 wrote: post #49

When developing ML models, you rarely train "just one". The article mentions that they explored a not-so-large hyper-parameter space (i.e. they trained multiple models with different parameters each). It would be interesting to know how long does the whole process takes on the M1 vs the V100. For the small models covered in the article, I'd guess that the V100 can train them all concurrently using MPS (multi-process…

The low gpu utilization rate in the first graph is kind of a tell... Seems like the M1 is a little bit worse than 40% of a v100?

If that's the case that would be very good. One can buy lots of M1 mac minis for the price of a V100..

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#92

Post by volta87 » Fri, Jan 15, 2021, 5:47 PM UTC

sradman wrote: post #65

volta87 wrote: post #45

Earlier quoted context omitted.

> We don’t have apples-to-apples benchmarks We do: https://mlperf.org/ Just run their benchmarks. Submitting your results there is a bit more complicated, because all results there are "verified" by independent entities. If you feel like your AI use case is not well represented by any of the MLPerf benchmarks, open a discussion thread about it, propose a new benchmark, etc. The set of benchmarks there increases all t…

> on top of the MLPerf Training and MLPerf Inference benchmark suites, we now have a new MLPerf HPC suite to capture ML of very large models. I think the challenge is selecting the tests that best represent the typical ML/DL use cases for the M1 and comparing it to an alternative such as the V100 using a common toolchain like Tensorflow. One of the problems that I see is that the optimizer/codegen of the toolchain is…

> The challenge is selecting the tests that best represent the typical ML/DL use cases for the M1 and comparing it to an alternative such as the V100 using a common toolchain like Tensorflow.

The benchmarks there are actual applications of ML, that people use to solve real world problems. To get a benchmark accepted you need to argue and convince people that the problem the benchmark solves must be solved by a lot of people, and that doing so burns enough cycles worldwide to be helpful to design ML hardware and software.

The hardware and software then gets developed to make solving these problems fast, which then in turns make real-world applications of ML fast.

Suggesting that the M1 is a solution, and now we just need to find a good problem that this solution solves well and add it there as a benchmark is the opposite to how mlperf works, and hardware vendors suggesting this is the reason mlperf exists. We already have common ML problems that a lot of people need to solve. Either the M1 is good at those or it isn't. If it isn't, it should become better at those. Being better at problems people don't want / need to solve does not help anybody.

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#93

Post by volta87 » Fri, Jan 15, 2021, 6:02 PM UTC

junipertea wrote: post #80

lukas wrote: post #71

Earlier quoted context omitted.

Do you really train more than one model at the same time on a single GPU? In my experience that's pretty unusual. I completely agree with your conclusion here.

I found training multiple models on same GPU hit other bottlenecks (mainly memory capacity/bandwidth) fast. I tend to train one model per GPU and just scale the number of computers. Also, if nothing else, we tend to push the models to fit the GPU memory.

Memory became less of an issue for me with V100, and isn't really an issue with A100, at least when quickly iterating for newer models, when the sizes are still relatively small.

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#94

Post by sdenton4 » Fri, Jan 15, 2021, 6:38 PM UTC

volta87 wrote: post #91

sdenton4 wrote: post #74

Earlier quoted context omitted.

The low gpu utilization rate in the first graph is kind of a tell... Seems like the M1 is a little bit worse than 40% of a v100?

If that's the case that would be very good. One can buy lots of M1 mac minis for the price of a V100..

Well, you can also get many RTX 3080's (~$700) for the price of a V100 (~$6000), and the RTX 3080's are faster: https://browser.geekbench.com/cuda-benchmarks

As I understand it, the V100 price is mostly artificial datacenter markup, enabled by lack of competition...