Benchmarking TensorFlow on Nvidia GeForce RTX 3090

Post by p1esk » Tue, Sep 29, 2020, 7:47 PM UTC

option wrote: post #62

Jhsto wrote: post #43

Earlier quoted context omitted.

Sure, but compared to the 3080 I'd say that the main deal is the bigger RAM for copying reason than the increased core count.

TIP: if you need to emulate a bigger batch with less RAM available - use gradient accumulation trick. Super easy to implement in Pytorch and it is already implemented as a single flag (accumulate_grad_batches) in Pytorch Lightning.

Post by nullifidian » Tue, Sep 29, 2020, 7:55 PM UTC

jjoonathan wrote: post #45

liuliu wrote: post #40

Seems to be good speedup overall relative to 2080 Ti (including FP16: see relatives 2080 Ti v Titan: https://lambdalabs.com/blog/2080-ti-deep-learning-benchmarks... ). This suggests we should see another Titan card that is even more expensive in the pipeline given the FP16 performance? Or maybe TF32 performance is going to be what NVIDIA promotes in this generation (only if they have better number than FP16?)?

Here's hoping for an A100 titan with un-nerfed FP64. The 3090 is twice as nerfed as previous generations, which were also bad at 1:32. Now it's 1:64 :(

Post by ls612 » Tue, Sep 29, 2020, 7:57 PM UTC

Karunamon wrote: post #55

TylerE wrote: post #50

Earlier quoted context omitted.

100% utilization is a pretty huge assumption. And if you ARE actually running it that hard, you'd better budget for fairly frequent replacement cards.

Wait, what? I used to mine crypto on GPU back when that was a profitable thing to do, which involves leaving the card maxed out for long periods of time.. no real damage. Are modern cards really so fragile you can expect them to die off under heavy use even if properly cooled and not overclocked?

Post by bitL » Tue, Sep 29, 2020, 8:01 PM UTC

ml_hardware wrote: post #76

binarymax wrote: post #75

Earlier quoted context omitted.

The 3090 RTX is also $1000 cheaper than the Titan, so there's that. It would be nice if there was a good way to express value per dollar. Perhaps in GLUE accuracy and training time.

Totally agree! I think 3090 could be a lot more cost effective for researchers to dabble with NLP. But it really grinds my gears when people post these misleading benchmarks... the 3090 is handicapped at half-rate tensor core performance while the Titan RTX is not. So if you're someone who does their work mainly in FP32, you will see improved performance with the 3090. On the other hand, if you are an FP16 speed demo…

Post by bitL » Tue, Sep 29, 2020, 8:05 PM UTC

wombatmobile wrote: post #14

valine wrote: post #2

Seems like a good speedup relative to the Titan, especially for the money. I’d be interested to see the performance relative to the 3080 though. There are obviously vram limitations with the 3080 but it would still be interesting to see the difference in raw compute performance. In games the 3090 only gives a 15% performance bump relative to the 3080. If that pattern holds for machine learning tasks there is probably…

If this isn't OT... Could you kindly advise what kind of computer would make sense to purchase to begin learning about ML? I was assuming I'd get a 3080. Should I get a case that could potentially house 2 x 3080's? Does the case require any special cooling considerations, or just whatever will fit the cards? What CPU would you get?