Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

wandb.ai

Post reply on HN
Print view View original

21–30 of 94 posts

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#21

Post by spi »

fxtentacle wrote: post #13
helsinkiandrew wrote: post #8

Can someone with more knowledge of Nvidia GPU's please say how much the V100 costs ($5-10K?) compared with the $900 mac mini.

You would instead buy a used 1080 (no ti) for similar performance. The special thing about the V100 is that it's driver EULA allows data center usage. If you don't need that, there are other much cheaper options.

"Similar performance" still means 30%-50% slower [1] and half the RAM, not really that comparable.

For much closer performance you should get a 2080ti, which should be roughly comparable in speed and have 11GB [edit: wrongly wrote 14GB before] of memory (against the 16GB for the V100). Price-wise you still save a lot of money, after quickly googling around, roughly $1200 vs. $15k-$20k.

But you still lose something, e.g. if you use half precision on V100 you get virtually double speed, if you do on a 1080 / 2080 you get... nothing because it's not supported.

(and more importantly for companies, you can actually use only V100-style stuff on servers [edit: as you mentioned already, although I'm not 100% sure it's just drivers that are the issue?])

[1] I've not used 1080 myself, but I've used 1080ti and V100 extensively, and the latter is about 30% faster. Hence my estimate for comparison with 1080

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#22

Post by fsh »

littlestymaar wrote: post #19
fxtentacle wrote: post #13

Earlier quoted context omitted.

You would instead buy a used 1080 (no ti) for similar performance. The special thing about the V100 is that it's driver EULA allows data center usage. If you don't need that, there are other much cheaper options.

> The special thing about the V100 is that it's driver EULA allows data center usage. Wait what? Is it the only thing? That sounds hard to believe: if true, using the open driver (Nouveau) instead of Nvidia's proprietary one would be a massive money saver for datacenters operators (and even if Nouveau doesn't support the features you'd want already, supporting their development would be much cheaper for a company lik…

Nouveau does not support CUDA and is therefore not usable for GPU computing on Nvidia.

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#23

Post by YetAnotherNick »

littlestymaar wrote: post #19
fxtentacle wrote: post #13

Earlier quoted context omitted.

You would instead buy a used 1080 (no ti) for similar performance. The special thing about the V100 is that it's driver EULA allows data center usage. If you don't need that, there are other much cheaper options.

> The special thing about the V100 is that it's driver EULA allows data center usage. Wait what? Is it the only thing? That sounds hard to believe: if true, using the open driver (Nouveau) instead of Nvidia's proprietary one would be a massive money saver for datacenters operators (and even if Nouveau doesn't support the features you'd want already, supporting their development would be much cheaper for a company lik…

NVIDIA has EULA to prevent data centre use of their hardware. Also, NVIDIA does not allow bulk buying of RTX series.

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#24

Post by YetAnotherNick »

fxtentacle wrote: post #5

"trainable_params 12,810" laughs (for comparison, GPT3: 175,000,000,000 parameters) Can Apple's M1 help you train tiny toy examples with no real-world relevance? You bet it can! Plus it looks like they are comparing Apples to Oranges ;) This seems to be 16 bit precision on the M1 and 32 bit on the V100. So the M1-trained model will most likely yield worse or unusable results, due to lack of precision. And lastly, the…

For the first graph:

  trainable parameters: 2236682

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#25

Post by dx034 »

sillysaurusx wrote: post #20
fxtentacle wrote: post #13

Earlier quoted context omitted.

You would instead buy a used 1080 (no ti) for similar performance. The special thing about the V100 is that it's driver EULA allows data center usage. If you don't need that, there are other much cheaper options.

Don't buy hardware in general for AI work, IMO. It'll be out of date in a year and you'll end up training in the cloud anyway.

If you properly utilize your hardware, on premise (or colocation in an area with cheap electricity prices) is vastly cheaper and will likely continue to be for a while. I don't see how training models in the cloud makes financial sense for organizations that can utilize their hardware 24/7.

For all others with burst workloads training in the cloud can make sense, but that has been the case for a while already.

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#27

Post by sillysaurusx »

dx034 wrote: post #25
sillysaurusx wrote: post #20

Earlier quoted context omitted.

Don't buy hardware in general for AI work, IMO. It'll be out of date in a year and you'll end up training in the cloud anyway.

If you properly utilize your hardware, on premise (or colocation in an area with cheap electricity prices) is vastly cheaper and will likely continue to be for a while. I don't see how training models in the cloud makes financial sense for organizations that can utilize their hardware 24/7. For all others with burst workloads training in the cloud can make sense, but that has been the case for a while already.

We're not talking about organizations, though. I don't agree with your premise, either. People aren't training models 24/7, so the idea that it's "vastly cheaper and will continue to be for a while" isn't true.

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#28

Post by FeepingCreature »

spi wrote: post #21
fxtentacle wrote: post #13

Earlier quoted context omitted.

You would instead buy a used 1080 (no ti) for similar performance. The special thing about the V100 is that it's driver EULA allows data center usage. If you don't need that, there are other much cheaper options.

"Similar performance" still means 30%-50% slower [1] and half the RAM, not really that comparable. For much closer performance you should get a 2080ti, which should be roughly comparable in speed and have 11GB [edit: wrongly wrote 14GB before] of memory (against the 16GB for the V100). Price-wise you still save a lot of money, after quickly googling around, roughly $1200 vs. $15k-$20k. But you still lose something, e…

How does AMD stuff like Radeon VII or MI100 hold up?

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#29

Post by YetAnotherNick »

fxtentacle wrote: post #14
apl wrote: post #12

Earlier quoted context omitted.

Hard disagree. V100s are a perfectly valid comparison point. They're usually what's available at scale (on AWS, in private clusters, etc.) because nobody's rolled out enough A100s at this point. If you look at any paper from OpenAI et al. (basically: not Google), you'll see performance numbers for large V100 clusters.

Yes and you'll see parameters tuned for V100, not parameters tuned for m1 somehow limping along on a V100 in emulation mode. I wouldn't complain about a benchmark executing any real world SOTA model on m1 and V100, but those will most likely not even run on the M1 due to memory constraints. So this article is like using an ios game to evaluate a Mac pro. You can do it, but it's not really useful.

You can count the number of GPUs having more than M1 memory(16 GB) in a single hand.

Re: Analyzing the performance of Tensorflow training on M1 Mac Mini and Nvidia V100

#30

Post by fxtentacle »

FeepingCreature wrote: post #28
spi wrote: post #21

Earlier quoted context omitted.

"Similar performance" still means 30%-50% slower [1] and half the RAM, not really that comparable. For much closer performance you should get a 2080ti, which should be roughly comparable in speed and have 11GB [edit: wrongly wrote 14GB before] of memory (against the 16GB for the V100). Price-wise you still save a lot of money, after quickly googling around, roughly $1200 vs. $15k-$20k. But you still lose something, e…

How does AMD stuff like Radeon VII or MI100 hold up?

Can't use it because most AI frameworks won't run on AMD because they did not implement suitable back-ends (yet).
Post reply on HN