Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Which GPU(s) to Get for Deep Learning

timdettmers.com

Post reply on HN
Print view View original

101–110 of 119 posts

Re: Which GPU(s) to Get for Deep Learning

#101

Post by jacquesm »

tensor wrote: post #85
boulos wrote: post #73

Earlier quoted context omitted.

Can you explain your math? A K80 on GCE is $.7/hr x 730 => $511/month if you were really 24x7. A K80 (and really we sell them by the die not the board) is more than $1000. I don't disagree that a consumer board is about that price, but they're not apples to apples. (Either in memory size, reliability or both). I'm fine with that being the real complaint: (major) cloud providers only sell the Tesla class boards, and t…

Sure, in looking at what it would cost to replicate this: https://arxiv.org/pdf/1704.01444.pdf They say they used four Titan X cards for a month. I was actually looking at google cloud machine learning since I thought being targeted to tensorflow would be the most cost effective. But it's 0.49/hour per ML training unit, or 1.47/hour per GPU. A basic gpu gives 3 training units, so I thought that something approximatel…

The comparison should be between a machine off-premises with as many GPUs in it of the same type that you intend to use and power + cooling.

Payback time according to my own calculations depending on GPU model and machine used compared to the various cloud offerings is between 4 and 6 months when used continuously.

Re: Which GPU(s) to Get for Deep Learning

#102

Post by mmusc »

fest wrote: post #91

I'm wondering about two things: 1. Can laptops with, say NVidia 1070 or 1080 GPUs keep them cool at their stock frequencies for a few hours? My work laptop with non-U i7 (Thinkpad T440p) starts thermal throttling in just a few minutes when I compile something large. 2. Wouldn't two NVidia 1060 or 1070 outperform a single 1080 for training, assuming batch size is kept low enough so that each batch fits in a single car…

1. Depends on the laptop really. Some have better cooling than others. I'm training models for a few hours every day with an asus rog that has a 1060, and I'm not experiencing any throttling.

Re: Which GPU(s) to Get for Deep Learning

#103

Post by technics256 »

marmaduke wrote: post #100
boulos wrote: post #73

Earlier quoted context omitted.

Can you explain your math? A K80 on GCE is $.7/hr x 730 => $511/month if you were really 24x7. A K80 (and really we sell them by the die not the board) is more than $1000. I don't disagree that a consumer board is about that price, but they're not apples to apples. (Either in memory size, reliability or both). I'm fine with that being the real complaint: (major) cloud providers only sell the Tesla class boards, and t…

> but they're not apples to apples It would be nice to see Nvidia or someone expand on this, so that users who have to make this choice can do so without guessing. If Google or AWS or M$ could publish reliability information, that'd be cool too. Illustrative case: I run Monte Carlo work on GPUs and administer a local compute cluster. I tested a workload on a 16 GB P100 and a GTX 1080. A 12 GB P100 costs (academic, EU…

Thank you for the answer. When you say performance difference is 2x, I presume the P100 is 2x faster than the 1080 in training epoch time?

Re: Which GPU(s) to Get for Deep Learning

#104

Post by dahauns »

throwaway32131 wrote: post #71
zeptomu wrote: post #27

There is an interesting offer by Hetzner that's available for some month now. They provide a dedicated server with a GTX 1080 for ~99e/month (111$/month) with adequate CPU (i7-6700), 64G Memory, 500G disk space and 50TB Traffic - there are also on-demand offerings by GCP and AWS, but I do not think they can match the offer by Hetzner: https://www.hetzner.de/us/hosting/produkte_rootserver/ex51ss... . Keep in mind that…

The electricity will likely cost a significant chunk of that, if one were running that full time in Germany. 0.5 * 24 * 30 * (30 c/kWH) = $108 However, it'd be a third of that in the states (may be cheaper in WA).

Industrial consumers in Germany pay nowhere near 30c/kwh, more like 10-15c. (depending on size and exemptions)

Re: Which GPU(s) to Get for Deep Learning

#105

Post by AlphaSite »

dragandj wrote: post #19
sp332 wrote: post #14

Earlier quoted context omitted.

I think signs are pointing to Vega being a HPC beast. But what they really need is the software ecosystem and support, and so far that hasn't been there. So while the new hardware looks cool, I'm really waiting for an announcement that OpenCL tooling got a lot better, or that CUDA is getting first-party support from AMD, anything to tempt those customers away from nvidia.

CUDA is getting support from AMD in the form of HIP. In my opinion, the problem is different: even if AMD supported CUDA proper, that still won't lead them anywhere because the reason people use Nvidia is not so much CUDA itself (although it has good tools) but Nvidia's proprietary libraries: cuBLAS, cuDNN, cuFFT etc. If AMD provided good implementations of similar libraries with OpenCL and shipped them with their dr…

They have rocBLAS, etc.

Re: Which GPU(s) to Get for Deep Learning

#106

Post by AlphaSite »

dogma1138 wrote: post #54
AlphaSite wrote: post #13

Interestingly, AMD is claiming that their upcoming Vega Frontier Editions chips outperform Nvidia's current P100 Deep Learning chip. http://hexus.net/media/uploaded/2017/5/30f5633b-1bbf-49b7-9f...

AMDs benchmarks are highly doubtful. DeepBench isn't a benchmark it's a benchmarking tool they haven't released what dataset nor what code were they running. Their SpecPerf View benchmarks were a total FUD they compared NVIDIAs consumer (GeForce) drivers against Radeon Pro drivers. The SPV benchmarks look impressive until you realize they are lower than a Quadro M5000 which is based on the same Maxwell chip that driv…

Deep learning doesn't use FP64 at all, does it?

Re: Which GPU(s) to Get for Deep Learning

#107

Post by zeptomu »

chronic940 wrote: post #86
zeptomu wrote: post #40

Earlier quoted context omitted.

It is not that bad. Either you've your training data local (unlikely) or it is already available in the "cloud" (i.e. a public-facing service). Let's say a typical training set (raw data) is 100G. Assuming 2.5MB/s Upload capacity with local data, you've uploaded it to your deep learning machine in half a day (100000 / 2.5 / 3600 ~ 11 hours) - which is not that much, as most of your time will be used for development a…

You forgot the AWS Internet egress charge per GB..

What do you mean?

I know egress is one of the more expensive cloud services (e.g. compared to compute and storgage) at AWS, GCP, etc., but if I upload data to my learning system that's ingress AFAIK which is mostly free or less expensive. Btw. current Egress is like 0.1$/GB, so 100G ~ 10$.

Don't get me wrong, I am not saying you should always train in the cloud, but I do not think slower Upload or Ingress are the limiting factor.

Re: Which GPU(s) to Get for Deep Learning

#108

Post by dogma1138 »

dragandj wrote: post #99
ktta wrote: post #95

Earlier quoted context omitted.

I haven't heard much about AMD's ACML and ArrayFire but I'm not surprised. On the other hand, I think you're underestimating the fact that OpenCL is an open spec and therefore has support from the FOSS world. CUDA has always been criticized for being closed source. Even though OpenCL doesn't get much praise, and actually get criticized a bit (I'm personally not a big fan because writing OpenCL code is like bending ov…

[1] is actually AMD's old libraries. Judging by dev activity, it seems to be abandonware in favor of https://github.com/RadeonOpenCompute which is itself not ready for prime time.

ROCm is an amalgamation of a lot of things, some new some (very) old.

It still relies on many pretty old libraries with questionable performance at best.

The newer stuff is good but it's tailored to AMD GPUs, don't expect interoperability when it comes to any reasonable performance if you want to run ROCm OpenCL on NVIDIA or Intel hardware.

Re: Which GPU(s) to Get for Deep Learning

#109

Post by dogma1138 »

ktta wrote: post #95
dogma1138 wrote: post #57

Earlier quoted context omitted.

The problem with OpenCL is the lack of well performing libraries, AMDs Core Math and Performance libraries are utter garbage still. If you can't do what Blender did and write everything from scratch including your primatives you'll be much slower than CUDA. But there is also a cost to it the Blender Cycles OpenCL code is nearly 5 times as big as their CUDA code and Cycles on OpenCL is still not at a feature parity wi…

I haven't heard much about AMD's ACML and ArrayFire but I'm not surprised. On the other hand, I think you're underestimating the fact that OpenCL is an open spec and therefore has support from the FOSS world. CUDA has always been criticized for being closed source. Even though OpenCL doesn't get much praise, and actually get criticized a bit (I'm personally not a big fan because writing OpenCL code is like bending ov…

CUDA is tricky, it's not FOSS but it's also not completely "closed" source either it's just a restricted license.

Many of the core libraries are not exactly open but these are primitives.

If say cuDNN would become open source today there won't be any benefit from it, you won't be able to make it run faster than what NVIDIA has already achieved.

As for the vendor lock well this is tricky, AMD also tried to vendor lock their initial compute APIs they've fallen back on OpenCL because it wasn't working.

ROCm isn't exactly vendor neutral while it can run on nearly any OpenCL compatible hardware it's tailored and optimized for AMD GPUs which means it would run like utter garbage on anything else.

It also supports interoperability with CUDA via the HIP compiler.

And in all honest at least for scientific computing OpenACC (https://www.openacc.org/) might make this argument irrelevant at least as far as general use code protability goes.

Re: Which GPU(s) to Get for Deep Learning

#110

Post by timdettmers »

boulos wrote: post #61

Disclaimer: I work on Google Cloud. I saw a lot of "should we use Cloud, no its crazy a GPU only costs $X". The key is that if you believe GPUs are going to get updated every year, and/or the best thing for ML may change (see TPU and plenty of startups with custom hardware) then suddenly buying hardware for 24? 36? months isn't as obvious. We (and AWS and Microsoft) have K80s because Maxwell wasn't a sufficiently fri…

The site is up again.

I have a prepared draft for a blog post exactly on the topic of cloud computing and deep learning. I did not finish it as I thought that there would not be much interest in the overall question since most people will just buy GTX cards. However, it seems that there is quite a confusion going on what makes sense and what does given certain circumstances. I think I will finish that blog post now and post it in the next days.

If you want me to discuss certain questions regarding deep learning hardware and cloud computing let me know here.

Post reply on HN