The Nvidia DGX-1 Deep Learning Supercomputer in a Box • Hacker News Board

Re: The Nvidia DGX-1 Deep Learning Supercomputer in a Box

#51

Post by dharma1 » Tue, Apr 05, 2016, 10:45 PM UTC

DeepYogurt wrote: post #40

Robadob wrote: post #32

Earlier quoted context omitted.

I haven't seen any recent benchmarks, but ones from 2011 all seemed to show CUDA and OpenCL on open footing in terms of performance when optimised properly.[1][2] CUDA simply had better library support, and a more well defined and uniform architecture to target. Whereas OpenCL is likely to require more programming to fill in the gaps for library support, and different optimisations depending on the architecture you w…

Would you mind trying the AMD compiler? http://gpuopen.com/compute-product/hip-convert-cuda-to-porta... I'd be interested in seeing a benchmark between some original cuda code and the opencl output of this compiler on the same gpu.

It doesn't look like AMD's HIP effort compiles to OpenCL, but to C++, that works on specific AMD GPUs (Fiji R9 Fury X, R9 Fury, R9 Nano).

Also, much of the speed gains for ML on NVidia hardware come from CuDNN - there is no equivalent for OpenCL or AMD hardware

Re: The Nvidia DGX-1 Deep Learning Supercomputer in a Box

#52

Post by blt » Tue, Apr 05, 2016, 10:47 PM UTC

phelm wrote: post #18

I am looking forward to OpenCL catching up with CUDA in maturity and adoption, so that NVidia's monopoly in Silicon for deep learning will come to an end.

I'm at GPU Technology Conference, where this computer was announced this morning. The amount of "wood behind the arrow" NVidia has for AI is insane. Even though the current demographic of GPU development is full of HPC simulations, physics, graphics... it's obvious that their biggest thrust is in machine learning. I don't think OpenCL can compete with this amount of money and enthusiasm. NVidia is rich and their engineers are very good. Some big changes would need to happen before OpenCL catches up to CUDA.

Re: The Nvidia DGX-1 Deep Learning Supercomputer in a Box

#53

Post by bgalbraith » Tue, Apr 05, 2016, 10:50 PM UTC

make3 wrote: post #36

bgalbraith wrote: post #31

Earlier quoted context omitted.

NVIDIA does not have a monopoly in the traditional sense. But yes, the have a de facto one because there is no viable competition. It's like saying MATLAB has a monopoly in academic research because so much of the code is written in it. That is slowly changing and moving over to Python now, which is great. Maybe OpenCL will get there someday, but I don't see it happening any time soon.

this is wrong. No mainstream deep learning library uses openCL, and the non-mainstream ones that do are much much slower. I remember reading up to 10x slower, but I can't seem to find the reference right now.

You are correct. My initial response was a pedantic point about the semantic use of monopoly in this context, which isn't helpful.

I would love it if AMD would care more about GPGPU, but they don't, and NVIDIA has little incentive to make their OpenCl drivers equal to their CUDA ones.

Re: The Nvidia DGX-1 Deep Learning Supercomputer in a Box

#54

Post by blakes » Tue, Apr 05, 2016, 10:51 PM UTC

manav wrote: post #50

$129k for this machine. In the keynote its interesting that they mentioned the product line being: "Tesla M40 for hyperscale, K80 for multi-app HPC, P100 for scales very high, and DGX-1 for the early adopters". The GP100/P100 with the 16nm process probably gives a considerable performance/power advantage over the Tesla... but this gives me the feeling that we may not see consumer or workstation-level Pascal boards fo…

$129k seem extremely fair for what you get actually, in my experience.

Re: The Nvidia DGX-1 Deep Learning Supercomputer in a Box

#55

Post by dharma1 » Tue, Apr 05, 2016, 10:52 PM UTC

blt wrote: post #52

phelm wrote: post #18

I am looking forward to OpenCL catching up with CUDA in maturity and adoption, so that NVidia's monopoly in Silicon for deep learning will come to an end.

I'm at GPU Technology Conference, where this computer was announced this morning. The amount of "wood behind the arrow" NVidia has for AI is insane. Even though the current demographic of GPU development is full of HPC simulations, physics, graphics... it's obvious that their biggest thrust is in machine learning. I don't think OpenCL can compete with this amount of money and enthusiasm. NVidia is rich and their engi…

I think it's just really bad management from AMD. Took them ages to wake up, and now they have what looks like a relatively small team on their Boltzmann initiative. Remains to be seen what happens to it.

How much do you think it would really cost to develop an OpenCL equivalent of CuDNN (even a stripped down version, just fast)? I know AMD are struggling but we are talking about allocating a handful of talented engineers

Re: The Nvidia DGX-1 Deep Learning Supercomputer in a Box

#56

Post by dougmany » Tue, Apr 05, 2016, 11:04 PM UTC

This announcement reminds me of the part of Outliers that spelled out how Bill Gates and others became who they are because they had access to very expensive equipment before anyone else did (and spent 10K hours on it).

Re: The Nvidia DGX-1 Deep Learning Supercomputer in a Box

#57

Post by aperrien » Tue, Apr 05, 2016, 11:08 PM UTC

Does anyone know if the Pascal architecture is built using stacked cores? Or is this one of those applications where thermal problems keep that technique from being used?

Re: The Nvidia DGX-1 Deep Learning Supercomputer in a Box

#58

Post by dharma1 » Tue, Apr 05, 2016, 11:09 PM UTC

dazzeruk wrote: post #3

chm wrote: post #2

Any idea how much this costs?

The Nvidia slides had it at $129,000 a pop

I'll take 5

Re: The Nvidia DGX-1 Deep Learning Supercomputer in a Box

#59

Post by nshm » Tue, Apr 05, 2016, 11:14 PM UTC

Looks like a research in machine learning will only be done in huge corporations. You'll need an amount of funding comparable to LHC.

Time to use better models like kernel ensembles, maybe they are not that accurate, but they are easier to train on a single CPU.

Re: The Nvidia DGX-1 Deep Learning Supercomputer in a Box

#60

Post by wmf » Tue, Apr 05, 2016, 11:33 PM UTC

aperrien wrote: post #57

Does anyone know if the Pascal architecture is built using stacked cores? Or is this one of those applications where thermal problems keep that technique from being used?

No, the Pascal GPU itself is not stacked. Die stacking makes almost no sense for processors.