Benchmarking TensorFlow on Nvidia GeForce RTX 3090 • Hacker News Board

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#101

Post by dogma1138 » Tue, Sep 29, 2020, 10:03 PM UTC

my123 wrote: post #29

bufo wrote: post #13

Earlier quoted context omitted.

Nvidia nerfed at the software level the FP16 performance to disincentivize people from using this card as a TITAN / datacenter ML card replacement.

It isn't at the software level, FP16 goes through the tensor cores on Turing onwards: https://www.anandtech.com/show/13973/nvidia-gtx-1660-ti-revi... )

The ALUs are capable of half precision regardless of the tensor cores and aren’t restricted.

For “tensor ops” in GeForce cards FP16 with FP32 accumulate is done at half rate so you don’t get double the performance which you do get in Quadro and Titan cards using the same die.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#102

Post by bitL » Tue, Sep 29, 2020, 10:18 PM UTC

ml_hardware wrote: post #100

bitL wrote: post #87

Earlier quoted context omitted.

What do you think about TF32 in 3090? Could it replace FP32 with 5x speedup?

I've done a lot of work in ML numerics, and I think TF32 is a completely safe drop-in for FP32 for ML workloads. NVIDIA seems to think so too, which is why on A100 it won't even be an option, it will be the default mode for any FP32 matrix multiplies. But on 3090, I don't think the speedup will be 5x, it should be closer to like 2x. The 3090 has 35.6 TF/s at TF32 and the Titan RTX has 16.3 TF/s at FP32. Once again I…

So basically no difference to FP32. That sounds very handicapped.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#103

Post by fomine3 » Wed, Sep 30, 2020, 12:32 AM UTC

motorcitycobra wrote: post #64

I thought I read Nvidia was nerfing the GeForce cards. Does this disprove it?

NVIDIA nerfs FP64 performance on consumer GeForce for recent years. It's critical for scientific calculations but not needed for ML. Alternatively they banned to run GeForce on datacenter.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#104

Post by bufo » Thu, Oct 01, 2020, 6:14 PM UTC

my123 wrote: post #29

bufo wrote: post #13

Earlier quoted context omitted.

Nvidia nerfed at the software level the FP16 performance to disincentivize people from using this card as a TITAN / datacenter ML card replacement.

It isn't at the software level, FP16 goes through the tensor cores on Turing onwards: https://www.anandtech.com/show/13973/nvidia-gtx-1660-ti-revi... )

See this thread https://twitter.com/wightmanr/status/1309583916362117120