Benchmarking TensorFlow on Nvidia GeForce RTX 3090

Post by Jhsto » Tue, Sep 29, 2020, 5:25 PM UTC

wnevets wrote: post #32

valine wrote: post #2

Seems like a good speedup relative to the Titan, especially for the money. I’d be interested to see the performance relative to the 3080 though. There are obviously vram limitations with the 3080 but it would still be interesting to see the difference in raw compute performance. In games the 3090 only gives a 15% performance bump relative to the 3080. If that pattern holds for machine learning tasks there is probably…

Most video games probably aren't going to make the most of all of the extra CUDA cores on the 3090. I'm assuming that helps alot with machine learning, can someone who knows for sure confirm?

Post by Jhsto » Tue, Sep 29, 2020, 5:35 PM UTC

option wrote: post #42

Jhsto wrote: post #41

Earlier quoted context omitted.

Most parallel processing scales linearly with core count. But the 3090 is more interesting for machine learning because of its RAM, which it has 24GB against 3080's 10GB. With machine learning, you spent most of the time copying memory between the CPU and GPU, so being able to fit more data to it reduces computation latency.

“ With machine learning, you spent most of the time copying memory between the CPU and GPU” - this is a sign that you are most likely doing it wrong. Yes, some operations are inherently bandwidth bound, but most important ones such as larger matrix multiplies (transformers) and convolutions are compute bound.

Post by jjoonathan » Tue, Sep 29, 2020, 5:41 PM UTC

liuliu wrote: post #40

Seems to be good speedup overall relative to 2080 Ti (including FP16: see relatives 2080 Ti v Titan: https://lambdalabs.com/blog/2080-ti-deep-learning-benchmarks... ). This suggests we should see another Titan card that is even more expensive in the pipeline given the FP16 performance? Or maybe TF32 performance is going to be what NVIDIA promotes in this generation (only if they have better number than FP16?)?

Post by p1esk » Tue, Sep 29, 2020, 5:49 PM UTC

jjoonathan wrote: post #45

liuliu wrote: post #40

Seems to be good speedup overall relative to 2080 Ti (including FP16: see relatives 2080 Ti v Titan: https://lambdalabs.com/blog/2080-ti-deep-learning-benchmarks... ). This suggests we should see another Titan card that is even more expensive in the pipeline given the FP16 performance? Or maybe TF32 performance is going to be what NVIDIA promotes in this generation (only if they have better number than FP16?)?

Here's hoping for an A100 titan with un-nerfed FP64. The 3090 is twice as nerfed as previous generations, which were also bad at 1:32. Now it's 1:64 :(

Post by jjoonathan » Tue, Sep 29, 2020, 5:52 PM UTC

gambiting wrote: post #28

wombatmobile wrote: post #14

Earlier quoted context omitted.

If this isn't OT... Could you kindly advise what kind of computer would make sense to purchase to begin learning about ML? I was assuming I'd get a 3080. Should I get a case that could potentially house 2 x 3080's? Does the case require any special cooling considerations, or just whatever will fit the cards? What CPU would you get?

If you're "learning about ML" there is no point in buying anything. Just get the cloud compute instead, and for home use and testing literally anything will do. I have friends who work with ML professionally and even they say it's just hard to justify running any computations at home once you factor in the electricity and hardware cost - GCP compute just beats the cost, easily.