Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Correcting Intel's Deep Learning Benchmark Mistakes

blogs.nvidia.com

Post reply on HN
Print view View original

21–30 of 40 posts

Re: Correcting Intel's Deep Learning Benchmark Mistakes

#21

Post by scottlegrand »

pavanky wrote: post #8
modeless wrote: post #5

> Titan uses four-year-old GPUs ... as does nearly every public cloud provider. I agree with most of the article, but you can't fault Intel for benchmarking the hardware that cloud providers are actually offering. I'm not sure what exactly NVIDIA is doing with their Tesla product line but whatever it is, it's really restricting the availability of recent GPU hardware. Even Azure's GPU instances released this month ar…

I mean Intel is comparing their publicly unavailable product against NVIDIA's publicly available product. Now NVIDIA is replying with the benchmarks on their (as of yet) publicly unavailable product. I think this blog post is fair game.

Not even wrong. I have two PCs with 4 Titan X (Maxwell) GPUs and a third PC with 4 Titan X (Pascal) GPUs. Both of these systems are available today (I built them myself, total BOM about $7K), and both will destroy 4 Xeon Phi servers at Deep Learning.

The benchmark Intel presented here is as disingenuous as their infamous white paper from 2010: http://pcl.intel-research.net/publications/isca319-lee.pdf

In comparison, a single Knights Landing Xeon Phi will be ~$7K. I know where I put my money. Caveat Emptor.

But Xeon Phi and I go way back here. They've been trying to beat my AMBER GPU code since 2013 or so. Many man years later I believe that a Knight's Corner is now ~35% faster than 2 Xeon CPUs with 1M atoms or more (source: http://adsabs.harvard.edu/abs/2016CoPhC.201...95N)

Meanwhile, the CUDA code has continued to scale with the GPU roadmap and a Titan XP is arguably 9-10x faster than 2 Xeon CPUs. No data is supplied at the low-end for Xeon Phi and I think we can safely assume it's because performance there sucks. (source: http://ambermd.org/gpus/benchmarks.htm)

Xeon Phi? IMO avoid avoid avoid until they start winning head to head 3rd party benchmarking fights like Soumith Chintala's fantastic convnet benchmark data: https://github.com/soumith/convnet-benchmarks

Re: Correcting Intel's Deep Learning Benchmark Mistakes

#22

Post by [deleted] »

modeless wrote: post #5

> Titan uses four-year-old GPUs ... as does nearly every public cloud provider. I agree with most of the article, but you can't fault Intel for benchmarking the hardware that cloud providers are actually offering. I'm not sure what exactly NVIDIA is doing with their Tesla product line but whatever it is, it's really restricting the availability of recent GPU hardware. Even Azure's GPU instances released this month ar…

[deleted]

Re: Correcting Intel's Deep Learning Benchmark Mistakes

#23

Post by mtgx »

sp332 wrote: post #18
wmf wrote: post #13

Earlier quoted context omitted.

I think both Knights Whatever and DGX-1 are already there. They seem to be in "order now, receive sometime" mode.

Xeon Phi is in 23 of the Top500 supercomputer list, so it's not like they're not shipping. The next version, Knight's Landing, should be launching soon, and hopefully will have better availability.

Intel was comparing Knights Landing to Kepler, which was released a few years back.

Re: Correcting Intel's Deep Learning Benchmark Mistakes

#25

Post by paulmd »

mtgx wrote: post #23
sp332 wrote: post #18

Earlier quoted context omitted.

Xeon Phi is in 23 of the Top500 supercomputer list, so it's not like they're not shipping. The next version, Knight's Landing, should be launching soon, and hopefully will have better availability.

Intel was comparing Knights Landing to Kepler, which was released a few years back.

Kepler was the default GPGPU product for many tasks until GP100 released - and GP100 still has not hit general availability yet.

Essentially, anything that needed dynamic parallelism (launching kernels from within kernels, i.e. tasks where you don't know where the difficult/interesting needles are within a haystack), advanced/concurrent scheduling capabilities, or FP64 is going to be much, much better off with Kepler until users can get their hands on GP100 cards. Maxwell is good at neither of those things - it's actually only good at specifically deep learning/neural nets. Which is not every task within the GPGPU space.

Re: Correcting Intel's Deep Learning Benchmark Mistakes

#26

Post by iofj »

NotQuantum wrote: post #12
ldargin wrote: post #11

I'm a bit surprised that Nvidia mentioned nothing about performance per watt in their reply.

Or that they didn't mention cost for performance. They equated 4 Xeon Phi servers to ONE DGX-1. The DGX has a 140k price tag.

How much are 4 Xeon Phi servers, mellanox cards and a large switch between them ?

Re: Correcting Intel's Deep Learning Benchmark Mistakes

#27

Post by throwaway287391 »

scottlegrand wrote: post #21
pavanky wrote: post #8

Earlier quoted context omitted.

I mean Intel is comparing their publicly unavailable product against NVIDIA's publicly available product. Now NVIDIA is replying with the benchmarks on their (as of yet) publicly unavailable product. I think this blog post is fair game.

Not even wrong. I have two PCs with 4 Titan X (Maxwell) GPUs and a third PC with 4 Titan X (Pascal) GPUs. Both of these systems are available today (I built them myself, total BOM about $7K), and both will destroy 4 Xeon Phi servers at Deep Learning. The benchmark Intel presented here is as disingenuous as their infamous white paper from 2010: http://pcl.intel-research.net/publications/isca319-lee.pdf In comparison,…

Yes to all of this. I'm really surprised they didn't compare costs in this blog post. Ignore the DGX-1 row of their table; the really damning comparison is between the 2nd and 4th rows of the table.

With a single 4x GPU server costing around $7k in total (row 4), you get nearly double the performance you get from spending $28k on four Xeon Phi servers (row 2).

And that's assuming you've spent the time and disk replicating your data on all four of those Xeon Phi servers, or went to a likely relatively large amount of engineering effort to ensure that network IO doesn't bottleneck training.

Re: Correcting Intel's Deep Learning Benchmark Mistakes

#29

Post by llukas »

paulmd wrote: post #25
mtgx wrote: post #23

Earlier quoted context omitted.

Intel was comparing Knights Landing to Kepler, which was released a few years back.

Kepler was the default GPGPU product for many tasks until GP100 released - and GP100 still has not hit general availability yet. Essentially, anything that needed dynamic parallelism (launching kernels from within kernels, i.e. tasks where you don't know where the difficult/interesting needles are within a haystack), advanced/concurrent scheduling capabilities, or FP64 is going to be much, much better off with Kepler…

Nvidia blog post is about Intel deep-learning benchmarks only.

Re: Correcting Intel's Deep Learning Benchmark Mistakes

#30

Post by fnord123 »

scottlegrand wrote: post #21
pavanky wrote: post #8

Earlier quoted context omitted.

I mean Intel is comparing their publicly unavailable product against NVIDIA's publicly available product. Now NVIDIA is replying with the benchmarks on their (as of yet) publicly unavailable product. I think this blog post is fair game.

Not even wrong. I have two PCs with 4 Titan X (Maxwell) GPUs and a third PC with 4 Titan X (Pascal) GPUs. Both of these systems are available today (I built them myself, total BOM about $7K), and both will destroy 4 Xeon Phi servers at Deep Learning. The benchmark Intel presented here is as disingenuous as their infamous white paper from 2010: http://pcl.intel-research.net/publications/isca319-lee.pdf In comparison,…

Omnipath also sucks compared to Infiniband. How are they making so many inroads into HPC with these offerings? I mean, aside from their dominant-for-good-reason CPUs.
Post reply on HN