Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Benchmarking TensorFlow on Nvidia GeForce RTX 3090

evolution.ai

Post reply on HN
Print view View original

51–60 of 104 posts

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#51

Post by Dylan16807 »

jjoonathan wrote: post #45
liuliu wrote: post #40

Seems to be good speedup overall relative to 2080 Ti (including FP16: see relatives 2080 Ti v Titan: https://lambdalabs.com/blog/2080-ti-deep-learning-benchmarks... ). This suggests we should see another Titan card that is even more expensive in the pipeline given the FP16 performance? Or maybe TF32 performance is going to be what NVIDIA promotes in this generation (only if they have better number than FP16?)?

Here's hoping for an A100 titan with un-nerfed FP64. The 3090 is twice as nerfed as previous generations, which were also bad at 1:32. Now it's 1:64 :(

The FP64 units are a separate addition that eat a lot of die space, right? I wouldn't use the word "nerf" for the tradeoff between having more SMs versus having more features in the SMs.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#52

Post by leeter »

posix_compliant wrote: post #23
valine wrote: post #21

Earlier quoted context omitted.

Well to start off I’m not advising buying two 3080s. I haven’t seen the benchmarks, and on top of that the 3080 doesn’t support SLI so if you do buy two of them you will need to be using software which can utilize two independent GPUs. If you’re just wanting to learn machine learning you don’t need anything particularly special. I think you would be happy with GTX 1070. There is also the cloud computing route where y…

Nvidia recommends 750W for the power supply.

It's worth noting that this recommendation has nothing to do with delivered power.... a 500W can handle most systems on delivered power.

This has to do with transients, the 3000 series cards have some massive transients that can easily trip OCP protection on powersupplies not designed for that kind of transient. A 700W powersupply is able to handle those transients much better than a 500W PSU is.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#53

Post by lsb »

jjoonathan wrote: post #45
liuliu wrote: post #40

Seems to be good speedup overall relative to 2080 Ti (including FP16: see relatives 2080 Ti v Titan: https://lambdalabs.com/blog/2080-ti-deep-learning-benchmarks... ). This suggests we should see another Titan card that is even more expensive in the pipeline given the FP16 performance? Or maybe TF32 performance is going to be what NVIDIA promotes in this generation (only if they have better number than FP16?)?

Here's hoping for an A100 titan with un-nerfed FP64. The 3090 is twice as nerfed as previous generations, which were also bad at 1:32. Now it's 1:64 :(

What workloads do you run that require fp64?

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#54

Post by jjoonathan »

TylerE wrote: post #50
jjoonathan wrote: post #49

Earlier quoted context omitted.

What's the (pre-Ampere) GCP price for a V100? On AWS it was $3/hr, so at 100% use and market prices a Titan V would pay for itself vs the cloud inside a month. Is GCP significantly cheaper? Or are we talking about pricing at ~0% utilization?

100% utilization is a pretty huge assumption. And if you ARE actually running it that hard, you'd better budget for fairly frequent replacement cards.

At 50% utilization it beats the cloud in 2 months. 10% utilization, it beats the cloud within a year. If you're dabbling, definitely go with the cloud, but if you're turning around experiments on a regular basis, buying gets attractive quickly.

And no, cards don't just keel over in a few months at 100%. Crypto miners ran that experiment. A typical card has years of 100% in it.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#55

Post by Karunamon »

TylerE wrote: post #50
jjoonathan wrote: post #49

Earlier quoted context omitted.

What's the (pre-Ampere) GCP price for a V100? On AWS it was $3/hr, so at 100% use and market prices a Titan V would pay for itself vs the cloud inside a month. Is GCP significantly cheaper? Or are we talking about pricing at ~0% utilization?

100% utilization is a pretty huge assumption. And if you ARE actually running it that hard, you'd better budget for fairly frequent replacement cards.

Wait, what? I used to mine crypto on GPU back when that was a profitable thing to do, which involves leaving the card maxed out for long periods of time.. no real damage.

Are modern cards really so fragile you can expect them to die off under heavy use even if properly cooled and not overclocked?

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#57

Post by opless »

danbr wrote: post #10

I just want to know how they installed the new nvidia cuda drivers without borking their Ubuntu/tf install.

NVidia drivers b0rked rebooting my box for a long time.

A couple of months ago, I removed all the references in apt sources, and followed the newer instructions (several times to get the right driver/cuda/tensorflow match) and my reboots are great, and only one GPU lock up so far (probably due to overheating - I've had to replace a couple of components flag as failed due to the heatwave in summer)

Jupyter hub is just great, I'd like to implement better diagnostics though ... have yet to find a good tutorial for that as yet.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#58

Post by oivey »

p1esk wrote: post #48
valine wrote: post #2

Seems like a good speedup relative to the Titan, especially for the money. I’d be interested to see the performance relative to the 3080 though. There are obviously vram limitations with the 3080 but it would still be interesting to see the difference in raw compute performance. In games the 3090 only gives a 15% performance bump relative to the 3080. If that pattern holds for machine learning tasks there is probably…

There will be 20GB version of 3080 soon.

There’s been rumors of this, but it’s not confirmed. A big percentage of the cost of the 3090 (and 3080) is the GDDR6X. A 3080 with 20GB of GDDR6X will still be really expensive, so it’s unclear to me that they will actually release something like that. Potentially they could put that much RAM on a card and then use slower GDDR6, but that’s kind of an odd part in Nvidia’s product offerings because then the 3080 with more RAM would be slower in a lot of situations.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#59

Post by jjoonathan »

Karunamon wrote: post #55
TylerE wrote: post #50

Earlier quoted context omitted.

100% utilization is a pretty huge assumption. And if you ARE actually running it that hard, you'd better budget for fairly frequent replacement cards.

Wait, what? I used to mine crypto on GPU back when that was a profitable thing to do, which involves leaving the card maxed out for long periods of time.. no real damage. Are modern cards really so fragile you can expect them to die off under heavy use even if properly cooled and not overclocked?

No, they are not that fragile.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#60

Post by jjoonathan »

lsb wrote: post #53
jjoonathan wrote: post #45

Earlier quoted context omitted.

Here's hoping for an A100 titan with un-nerfed FP64. The 3090 is twice as nerfed as previous generations, which were also bad at 1:32. Now it's 1:64 :(

What workloads do you run that require fp64?

Simulation.
Post reply on HN