Live data from Hacker News

Benchmarking TensorFlow on Nvidia GeForce RTX 3090

evolution.ai

11–20 of 104 posts

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#11
Can someone explain the difference between fp16 and fp32 in these benchmarks because the difference is pretty dramatic. I assume it's floating point precision(?) but why would lower precision be slower relatively on the 3090? For training jobs how does the precision impact accuracy of the model?

Edit: clarified that I am referring to slower relative performance

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#12
post #11

Can someone explain the difference between fp16 and fp32 in these benchmarks because the difference is pretty dramatic. I assume it's floating point precision(?) but why would lower precision be slower relatively on the 3090? For training jobs how does the precision impact accuracy of the model? Edit: clarified that I am referring to slower relative performance

FP16 is faster (units are images per second)

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#13
post #11

Can someone explain the difference between fp16 and fp32 in these benchmarks because the difference is pretty dramatic. I assume it's floating point precision(?) but why would lower precision be slower relatively on the 3090? For training jobs how does the precision impact accuracy of the model? Edit: clarified that I am referring to slower relative performance

Nvidia nerfed at the software level the FP16 performance to disincentivize people from using this card as a TITAN / datacenter ML card replacement.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#14
post #2

Seems like a good speedup relative to the Titan, especially for the money. I’d be interested to see the performance relative to the 3080 though. There are obviously vram limitations with the 3080 but it would still be interesting to see the difference in raw compute performance. In games the 3090 only gives a 15% performance bump relative to the 3080. If that pattern holds for machine learning tasks there is probably…

If this isn't OT...

Could you kindly advise what kind of computer would make sense to purchase to begin learning about ML? I was assuming I'd get a 3080. Should I get a case that could potentially house 2 x 3080's? Does the case require any special cooling considerations, or just whatever will fit the cards? What CPU would you get?

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#15
post #11

Can someone explain the difference between fp16 and fp32 in these benchmarks because the difference is pretty dramatic. I assume it's floating point precision(?) but why would lower precision be slower relatively on the 3090? For training jobs how does the precision impact accuracy of the model? Edit: clarified that I am referring to slower relative performance

Fp16 is faster in this article on most models...

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#17
post #2

Seems like a good speedup relative to the Titan, especially for the money. I’d be interested to see the performance relative to the 3080 though. There are obviously vram limitations with the 3080 but it would still be interesting to see the difference in raw compute performance. In games the 3090 only gives a 15% performance bump relative to the 3080. If that pattern holds for machine learning tasks there is probably…

If this isn't OT... Could you kindly advise what kind of computer would make sense to purchase to begin learning about ML? I was assuming I'd get a 3080. Should I get a case that could potentially house 2 x 3080's? Does the case require any special cooling considerations, or just whatever will fit the cards? What CPU would you get?

I'd honestly start with cloud options if learning is the only reason you're building the computer. You don't want to dump a bunch of money into depreciating GPUs if you're not going to end up using them.

GPUs are only really required in ML if you want to do deep neural network stuff. You can do plenty in CPU on reasonable data sets using any modern laptop.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#18
post #11

Can someone explain the difference between fp16 and fp32 in these benchmarks because the difference is pretty dramatic. I assume it's floating point precision(?) but why would lower precision be slower relatively on the 3090? For training jobs how does the precision impact accuracy of the model? Edit: clarified that I am referring to slower relative performance

[deleted]

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#20
post #11

Can someone explain the difference between fp16 and fp32 in these benchmarks because the difference is pretty dramatic. I assume it's floating point precision(?) but why would lower precision be slower relatively on the 3090? For training jobs how does the precision impact accuracy of the model? Edit: clarified that I am referring to slower relative performance

FP16 is faster (units are images per second)

Sorry I was referring to the relative performance, I edited my question to be clearer
Post reply on HN