I do not usually believe in any vendor provided performance benchmark unless I fabricated it myself. On a more serious note, benchmarking is pretty hard and people usually discount things that seem minor until you found it the bottleneck, like the interconnect in this case for example. Another problem with synthetic benchmarks is that you can always optimize for your exact use case and it usually yields to pretty good improvements, comparable to buying faster equipment. The ultimate quiestion which is more cost efficient, buying faster CPU/GPU or hiring a performance expert.
Correcting Intel's Deep Learning Benchmark Mistakes
31–40 of 40 posts
Re: Correcting Intel's Deep Learning Benchmark Mistakes
#32> Titan uses four-year-old GPUs ... as does nearly every public cloud provider. I agree with most of the article, but you can't fault Intel for benchmarking the hardware that cloud providers are actually offering. I'm not sure what exactly NVIDIA is doing with their Tesla product line but whatever it is, it's really restricting the availability of recent GPU hardware. Even Azure's GPU instances released this month ar…
True, however the cloud vendors do not have Intel latest greatest hardware either, so than comparing latest Intel Phi to some older Nvidia GPU does not make sense.
Re: Correcting Intel's Deep Learning Benchmark Mistakes
#33Why don't they provide a link to their testing methodology? They need to back up their claims (on both sides) with the actual configuration, all versions, and sample datasets for people to independently verify. A docker container that runs their performance suite would be ideal.
If you would like to measure Docker overhead it would perfect. Unfortunately not all the world is using Docker for everything so running performance tests using the least amount of complexity is usually better for these tests.
Re: Correcting Intel's Deep Learning Benchmark Mistakes
#34Ok, its officially the new new thing when corporate communications types are sniping at each other :-) Once we start seeing press releases of products that are not available yet being shown to kill the existing competition we'll know that the hype train has officially gone super-product-sonic (that is a hype wave travelling faster than the product releases can support it).
I think both Knights Whatever and DGX-1 are already there. They seem to be in "order now, receive sometime" mode.
Knights Landing has been received in the UK, at least.
Re: Correcting Intel's Deep Learning Benchmark Mistakes
#35Earlier quoted context omitted.
I mean Intel is comparing their publicly unavailable product against NVIDIA's publicly available product. Now NVIDIA is replying with the benchmarks on their (as of yet) publicly unavailable product. I think this blog post is fair game.
Not even wrong. I have two PCs with 4 Titan X (Maxwell) GPUs and a third PC with 4 Titan X (Pascal) GPUs. Both of these systems are available today (I built them myself, total BOM about $7K), and both will destroy 4 Xeon Phi servers at Deep Learning. The benchmark Intel presented here is as disingenuous as their infamous white paper from 2010: http://pcl.intel-research.net/publications/isca319-lee.pdf In comparison,…
So, a specific MD code may or may not work well with KNL -- we don't have data. KNL looks quite attractive for other chemistry, given all the vector units, large amount of fast memory, and ability to run realistically-sized examples without the network, or potentially the network-on-chip. We'll see how it pans out.
Re: Correcting Intel's Deep Learning Benchmark Mistakes
#36Earlier quoted context omitted.
Not even wrong. I have two PCs with 4 Titan X (Maxwell) GPUs and a third PC with 4 Titan X (Pascal) GPUs. Both of these systems are available today (I built them myself, total BOM about $7K), and both will destroy 4 Xeon Phi servers at Deep Learning. The benchmark Intel presented here is as disingenuous as their infamous white paper from 2010: http://pcl.intel-research.net/publications/isca319-lee.pdf In comparison,…
Omnipath also sucks compared to Infiniband. How are they making so many inroads into HPC with these offerings? I mean, aside from their dominant-for-good-reason CPUs.
Please share the data, particularly for the built-in interfaces on ~70 cores which are supposed to be available this year (given the blanket statement). Omni-path seems to be worrying Mellanox, judging by a recent visit.
Re: Correcting Intel's Deep Learning Benchmark Mistakes
#37Earlier quoted context omitted.
Not even wrong. I have two PCs with 4 Titan X (Maxwell) GPUs and a third PC with 4 Titan X (Pascal) GPUs. Both of these systems are available today (I built them myself, total BOM about $7K), and both will destroy 4 Xeon Phi servers at Deep Learning. The benchmark Intel presented here is as disingenuous as their infamous white paper from 2010: http://pcl.intel-research.net/publications/isca319-lee.pdf In comparison,…
So, a specific MD code may or may not work well with KNL -- we don't have data. KNL looks quite attractive for other chemistry, given all the vector units, large amount of fast memory, and ability to run realistically-sized examples without the network, or potentially the network-on-chip. We'll see how it pans out.
I prefer to look at it the other way, why don't you point out an existing and important chemistry application where KNL bested its contemporary GPUs, say the best of Knight's Corner versus the best of Kepler (K40 or K80). I'm also open to Knight's Landing versus GP100 (vaporware versus no longer vaporware but hard to get)
I'm genuinely interested here because I can't find this anywhere. I don't think it exists personally.
Re: Correcting Intel's Deep Learning Benchmark Mistakes
#38Earlier quoted context omitted.
So, a specific MD code may or may not work well with KNL -- we don't have data. KNL looks quite attractive for other chemistry, given all the vector units, large amount of fast memory, and ability to run realistically-sized examples without the network, or potentially the network-on-chip. We'll see how it pans out.
I prefer to look at it the other way, why don't you point out an existing and important chemistry application where KNL bested its contemporary GPUs, say the best of Knight's Corner versus the best of Kepler (K40 or K80). I'm also open to Knight's Landing versus GP100 (vaporware versus no longer vaporware but hard to get) I'm genuinely interested here because I can't find this anywhere. I don't think it exists person…
KNL is not Knights Corner, and I have limited information on either. I'm interested in data and, more to the point, insight -- not just single benchmark numbers or specific programs, especially if they've had a lot of GPU effort and no tuning for KNL. I don't expect KNL to be particularly good for applications that aren't highly vectorizable, though the memory system may help.
If I manage to access the KNL here, I'll probably run cp2k and gromacs, though single node performance is of limited interest, and ELPA doesn't currently have AVX512-specific support.
Re: Correcting Intel's Deep Learning Benchmark Mistakes
#39Earlier quoted context omitted.
I prefer to look at it the other way, why don't you point out an existing and important chemistry application where KNL bested its contemporary GPUs, say the best of Knight's Corner versus the best of Kepler (K40 or K80). I'm also open to Knight's Landing versus GP100 (vaporware versus no longer vaporware but hard to get) I'm genuinely interested here because I can't find this anywhere. I don't think it exists person…
KNL is not Knights Corner, and I have limited information on either. I'm interested in data and, more to the point, insight -- not just single benchmark numbers or specific programs, especially if they've had a lot of GPU effort and no tuning for KNL. I don't expect KNL to be particularly good for applications that aren't highly vectorizable, though the memory system may help. If I manage to access the KNL here, I'll…
Here's your answer for GROMACS (and it sucks)...
http://www.prace-ri.eu/IMG/pdf/wp120.pdf
Even so, right now, little would please me more technologically than a competitive Xeon Phi offering, but while KNL is better than KNC, my inside info says it sucks too (it would have been a lot more interesting, just like Altera's Stratix 10, if it had shipped before GP100 and GP102).
Right now, I have more confidence in AMD GPUs right now than I have in Xeon Phi. This 3rd party benchmark is particularly interesting (and it doesn't look like anyone at NVIDIA is paying any attention to it):
https://techaltar.com/amd-rx-480-gpu-review/2/
Sure, NVIDIA is still in the lead, but not with the ~10x margins they used to have over AMD.
Finally, I figuratively feel like punching the next person who makes the BS scaling argument over raw performance. GPUs scale too if they're coded correctly. And cloud datacenters are the worst place for that given their craptastic ~10 Gb/s interconnect subject to arbitrary network weather effects.
Or butchering Seymour Cray: Your life depends on winning a race, would you bet your life on a 1,350 HP Venom GT or on 20 179 HP Scion FRSs? I mean collectively that's almost 3600 HP, right? Except it's even worse because for GPUs vs CPUs, it's like they priced the Scion FRS like a Venom GT and vice versa.
I wish you luck finding Xeon Phi winning anything but synthetic tests against yesterday's news:
https://www.xcelerit.com/computing-benchmarks/libor/intel-xe...
Re: Correcting Intel's Deep Learning Benchmark Mistakes
#40They do not mention the version of Caffe used to test the Intel systems, Intel claims its numbers based on an optimized branch of Caffe and not the public (BVLC) version