Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Which GPU(s) to Get for Deep Learning

timdettmers.com

Post reply on HN
Print view View original

51–60 of 119 posts

Re: Which GPU(s) to Get for Deep Learning

#51

Post by jacquesm »

locusm wrote: post #46

How hard does the GPU work on these tasks? Is it Bitcoin mining level power consumption?

All out. This is typical:

  nvidia-smi
  Tue May 23 01:41:58 2017       
  +-----------------------------------------------------------------------------+
  | NVIDIA-SMI 375.39                 Driver Version: 375.39                    |
  |-------------------------------+----------------------+----------------------+
  | GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
  | Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
  |===============================+======================+======================|
  |   0  Graphics Device     Off  | 0000:01:00.0      On |                  N/A |
  | 57%   84C    P2   257W / 250W |  11010MiB / 11171MiB |     95%      Default |
  +-------------------------------+----------------------+----------------------+
That's during training. I'm running a minimal desktop to have maximum ram for the minibatch (and it still isn't enough but it will have to do).

Re: Which GPU(s) to Get for Deep Learning

#52

Post by solomatov »

ktta wrote: post #48

If anyone here doesn't want to spend money on $500+ GPU (or the $1k+ ones!), then I'd suggest getting the lowest tier Nvidia GPU for $100~[1] If that GPU is a real bottleneck for you, then you're much better off spending money on GCP/AWS's GPU offerings. That's because consumer GPUs get superseeded every year and online offering's price will only go down. So you can spend 10% of $1k every year and keep getting better…

>If that GPU is a real bottleneck for you, then you're much better off spending money on GCP/AWS's GPU offerings. That's because consumer GPUs get superseeded every year and online offering's price will only go down.

GCP and AWS have old GPUs and they are really really expensive. If you expect to run workloads for a long time, it would be more cost efficient to buy your own hardware.

Re: Which GPU(s) to Get for Deep Learning

#53

Post by solomatov »

KerrickStaley wrote: post #42

Question: if I'm learning about neural networks and want to e.g. train a network to recognize MNIST digits, do I need a discrete graphics card (probably attached to a VPS that I would rent)? Or can I use the i5 Kaby Lake (which has an integrated GPU) in my laptop to train my network?

Simple network form tensroflow's MNIST tutorial trains in about 30 minutes on my macbook pro.

Re: Which GPU(s) to Get for Deep Learning

#54

Post by dogma1138 »

AlphaSite wrote: post #13

Interestingly, AMD is claiming that their upcoming Vega Frontier Editions chips outperform Nvidia's current P100 Deep Learning chip. http://hexus.net/media/uploaded/2017/5/30f5633b-1bbf-49b7-9f...

AMDs benchmarks are highly doubtful. DeepBench isn't a benchmark it's a benchmarking tool they haven't released what dataset nor what code were they running.

Their SpecPerf View benchmarks were a total FUD they compared NVIDIAs consumer (GeForce) drivers against Radeon Pro drivers.

The SPV benchmarks look impressive until you realize they are lower than a Quadro M5000 which is based on the same Maxwell chip that drives the 980ti.

NVIDIA's consumer drivers (and the non Pro AMD ones) are simply horrible for CAD and other professional workloads.

AMD has been touting better performance than NVIDIA GPUs in compute since Fiji and they do not deliver.

It's also important to note that VEGA will have FP64 performance at 1/32 while the P100 is at 1/2.

I'm sure you can find an OpenCL workload that NVIDIA would be utterly trash in because the code is not optimized heck working with 256 batches as recommended for AMD rather than 1024 as recommended for NVIDIA would achive that alone.

Re: Which GPU(s) to Get for Deep Learning

#56

Post by KaoruAoiShiho »

Sephr wrote: post #43

Here's a much more cost effective answer: Use https://cloud.google.com/tpu/ Unless you have an unlimited supply of free electricity and don't care about the increased hardware management overhead, it's a waste of money to buy Pascal GPUs for large-scale deep learning. The following cards have much more optimized deep learning silicon and are publicly available /right now/: - Nvidia Tesla V100 (Tensor cores only: 120…

That makes no sense as the TPU2 is not really out yet for actual consumers. AFAIK it's only in closed alpha right now, so if you're actually doing stuff right now it's not a real option. There's also no pricing so the "cost effective" remark can't really be seen.

Re: Which GPU(s) to Get for Deep Learning

#57

Post by dogma1138 »

ktta wrote: post #47
sp332 wrote: post #14

Earlier quoted context omitted.

I think signs are pointing to Vega being a HPC beast. But what they really need is the software ecosystem and support, and so far that hasn't been there. So while the new hardware looks cool, I'm really waiting for an announcement that OpenCL tooling got a lot better, or that CUDA is getting first-party support from AMD, anything to tempt those customers away from nvidia.

You might be interested in knowing that OpenCL is merging with Vulkan's API[1], so there'll likely be a lot more support in the future. [1]: https://news.ycombinator.com/item?id=14383296

The problem with OpenCL is the lack of well performing libraries, AMDs Core Math and Performance libraries are utter garbage still.

If you can't do what Blender did and write everything from scratch including your primatives you'll be much slower than CUDA.

But there is also a cost to it the Blender Cycles OpenCL code is nearly 5 times as big as their CUDA code and Cycles on OpenCL is still not at a feature parity with CUDA.

The change in branding would likely not yield much for it.

Re: Which GPU(s) to Get for Deep Learning

#58

Post by maksimum »

arnon wrote: post #31
chronic940 wrote: post #12

Earlier quoted context omitted.

K80s and K40s have a K. That's two generations old. We are currently on P and about to be V. If you think K80s are good, you are far far behind the times in machine learning. A single 1080Ti even in a 4U server outperforms 2 K80s.

If you're running a rackmount server, you need the Tesla series. We found that the GeForces tend to burn out when under heavy load, whereas we've not had a single Tesla series ever burn out.

Ooh ooh tell us more. Which GeForces and which server chassis? Adequate power supply?

Re: Which GPU(s) to Get for Deep Learning

#59

Post by marvy »

KerrickStaley wrote: post #42

Question: if I'm learning about neural networks and want to e.g. train a network to recognize MNIST digits, do I need a discrete graphics card (probably attached to a VPS that I would rent)? Or can I use the i5 Kaby Lake (which has an integrated GPU) in my laptop to train my network?

As other people have said, for MNIST, CPU is fine. One core. You can get over 90% accuracy with a network with just 1 hidden layer with only a few seconds of training. Or something like that. If you've never played with MNIST before, it's kind of amazing how easy it is. For instance, the following idea "works", in that you get results that are pretty bad, but much better than chance. (Even more than 50% right, I think.) Suppose you have 10 arrays:

zeros: each element of this array is a picture of a "0" ones: each element is a picture of a "1" ... nines: each element is a "9"

Compute the average of each array: the "average" 0, the average 1, and so on. Then classify new digits based on which average element they are closest to, using Euclidean distance. This sounds way too dumb to do any good, but the MNIST digits are normalized so well that this actually does something.

Even better: you can have a neural "network" that has zero hidden layers. This actually achieves almost respectable performance, believe it or not.

Re: Which GPU(s) to Get for Deep Learning

#60

Post by dkobran »

GTX drivers become crippled when the card detects the presence of a virtual environment. This means you can't run GTX in the cloud, otherwise, cloud GPU prices would be much lower. Without the availability of GTX, we've been trying just about everything at Paperspace to bring prices down and make the cloud a viable option for GPU compute. The argument being, there are real benefits to running in the cloud like on-demand scalability, lack of upkeep, minimal upfront costs, and of course, running a production application :) There are other indirect cost saving eg power consumption which can be quite significant when training models for long periods of time. Would love to see a total cost of ownership figure added to this post.
Post reply on HN