Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Benchmarking TensorFlow on Nvidia GeForce RTX 3090

evolution.ai

Post reply on HN
Print view View original

21–30 of 104 posts

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#21

Post by valine »

wombatmobile wrote: post #14
valine wrote: post #2

Seems like a good speedup relative to the Titan, especially for the money. I’d be interested to see the performance relative to the 3080 though. There are obviously vram limitations with the 3080 but it would still be interesting to see the difference in raw compute performance. In games the 3090 only gives a 15% performance bump relative to the 3080. If that pattern holds for machine learning tasks there is probably…

If this isn't OT... Could you kindly advise what kind of computer would make sense to purchase to begin learning about ML? I was assuming I'd get a 3080. Should I get a case that could potentially house 2 x 3080's? Does the case require any special cooling considerations, or just whatever will fit the cards? What CPU would you get?

Well to start off I’m not advising buying two 3080s. I haven’t seen the benchmarks, and on top of that the 3080 doesn’t support SLI so if you do buy two of them you will need to be using software which can utilize two independent GPUs.

If you’re just wanting to learn machine learning you don’t need anything particularly special. I think you would be happy with GTX 1070. There is also the cloud computing route where you basically rent the gpu from AWS. That will be initially more cost effective than buying your own hardware.

One thing to keep in mind if you do go with the 3080 is the power consumption. Ampere cards are going to be much more power hungry than previous generations, and you will need to budget about 320W just to the graphics card. The recommended power supply for the 3080 is 850W.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#22

Post by bufo »

Der_Einzige wrote: post #15
bryan0 wrote: post #11

Can someone explain the difference between fp16 and fp32 in these benchmarks because the difference is pretty dramatic. I assume it's floating point precision(?) but why would lower precision be slower relatively on the 3090? For training jobs how does the precision impact accuracy of the model? Edit: clarified that I am referring to slower relative performance

Fp16 is faster in this article on most models...

That's because of the improved memory bandwidth. See https://timdettmers.com/2020/09/07/which-gpu-for-deep-learni...

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#23

Post by posix_compliant »

valine wrote: post #21
wombatmobile wrote: post #14

Earlier quoted context omitted.

If this isn't OT... Could you kindly advise what kind of computer would make sense to purchase to begin learning about ML? I was assuming I'd get a 3080. Should I get a case that could potentially house 2 x 3080's? Does the case require any special cooling considerations, or just whatever will fit the cards? What CPU would you get?

Well to start off I’m not advising buying two 3080s. I haven’t seen the benchmarks, and on top of that the 3080 doesn’t support SLI so if you do buy two of them you will need to be using software which can utilize two independent GPUs. If you’re just wanting to learn machine learning you don’t need anything particularly special. I think you would be happy with GTX 1070. There is also the cloud computing route where y…

Nvidia recommends 750W for the power supply.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#24

Post by kevingadd »

valine wrote: post #2

Seems like a good speedup relative to the Titan, especially for the money. I’d be interested to see the performance relative to the 3080 though. There are obviously vram limitations with the 3080 but it would still be interesting to see the difference in raw compute performance. In games the 3090 only gives a 15% performance bump relative to the 3080. If that pattern holds for machine learning tasks there is probably…

I think for high throughput scenarios the 3090 probably has more headroom due to its higher TDP and better (larger) cooling solution, which might really matter here if you're driving the tensor cores at max the whole time.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#25

Post by paol »

wombatmobile wrote: post #14
valine wrote: post #2

Seems like a good speedup relative to the Titan, especially for the money. I’d be interested to see the performance relative to the 3080 though. There are obviously vram limitations with the 3080 but it would still be interesting to see the difference in raw compute performance. In games the 3090 only gives a 15% performance bump relative to the 3080. If that pattern holds for machine learning tasks there is probably…

If this isn't OT... Could you kindly advise what kind of computer would make sense to purchase to begin learning about ML? I was assuming I'd get a 3080. Should I get a case that could potentially house 2 x 3080's? Does the case require any special cooling considerations, or just whatever will fit the cards? What CPU would you get?

If you're just beginning a 3080 is overkill, never mind two. Get a used 1080ti (or even 2080ti if not overpriced), it'll be cheaper and even has a bit more RAM.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#26

Post by secabeen »

posix_compliant wrote: post #23
valine wrote: post #21

Earlier quoted context omitted.

Well to start off I’m not advising buying two 3080s. I haven’t seen the benchmarks, and on top of that the 3080 doesn’t support SLI so if you do buy two of them you will need to be using software which can utilize two independent GPUs. If you’re just wanting to learn machine learning you don’t need anything particularly special. I think you would be happy with GTX 1070. There is also the cloud computing route where y…

Nvidia recommends 750W for the power supply.

That is true, but it all depends on how much the rest of your system uses. If you just have a mid-tier CPU and SSD, you may find that the rest of your system only uses 200W, and you can fit a 3080 on a 600W power supply. There are multiple reports of high-quality 600W PSUs working, so YMMV.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#27

Post by paol »

danbr wrote: post #10

I just want to know how they installed the new nvidia cuda drivers without borking their Ubuntu/tf install.

Nvidia has official ppa's with all versions of CUDA, libcudnn and drivers. If you install from there you will not have problems.

It helps keeping to Ubuntu LTS versions though, that's what they support best.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#28

Post by gambiting »

wombatmobile wrote: post #14
valine wrote: post #2

Seems like a good speedup relative to the Titan, especially for the money. I’d be interested to see the performance relative to the 3080 though. There are obviously vram limitations with the 3080 but it would still be interesting to see the difference in raw compute performance. In games the 3090 only gives a 15% performance bump relative to the 3080. If that pattern holds for machine learning tasks there is probably…

If this isn't OT... Could you kindly advise what kind of computer would make sense to purchase to begin learning about ML? I was assuming I'd get a 3080. Should I get a case that could potentially house 2 x 3080's? Does the case require any special cooling considerations, or just whatever will fit the cards? What CPU would you get?

If you're "learning about ML" there is no point in buying anything. Just get the cloud compute instead, and for home use and testing literally anything will do. I have friends who work with ML professionally and even they say it's just hard to justify running any computations at home once you factor in the electricity and hardware cost - GCP compute just beats the cost, easily.

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#29

Post by my123 »

bufo wrote: post #13
bryan0 wrote: post #11

Can someone explain the difference between fp16 and fp32 in these benchmarks because the difference is pretty dramatic. I assume it's floating point precision(?) but why would lower precision be slower relatively on the 3090? For training jobs how does the precision impact accuracy of the model? Edit: clarified that I am referring to slower relative performance

Nvidia nerfed at the software level the FP16 performance to disincentivize people from using this card as a TITAN / datacenter ML card replacement.

It isn't at the software level, FP16 goes through the tensor cores on Turing onwards: https://www.anandtech.com/show/13973/nvidia-gtx-1660-ti-revi...)

Re: Benchmarking TensorFlow on Nvidia GeForce RTX 3090

#30

Post by claudeganon »

gambiting wrote: post #28
wombatmobile wrote: post #14

Earlier quoted context omitted.

If this isn't OT... Could you kindly advise what kind of computer would make sense to purchase to begin learning about ML? I was assuming I'd get a 3080. Should I get a case that could potentially house 2 x 3080's? Does the case require any special cooling considerations, or just whatever will fit the cards? What CPU would you get?

If you're "learning about ML" there is no point in buying anything. Just get the cloud compute instead, and for home use and testing literally anything will do. I have friends who work with ML professionally and even they say it's just hard to justify running any computations at home once you factor in the electricity and hardware cost - GCP compute just beats the cost, easily.

Agreed. If you’re just learning or building hobby stuff, you can use Colab, Paperspace, or any number of other services for free or very cheaply.
Post reply on HN