HPC is dying, and MPI is killing it

I came into the essay with suspicion. A map-reduce system like Hadoop isn't a good fit for HPC problems, and I thought it would argue that MPI is old => it's stuck in the past.

Instead, and to my joy, it was a well-reasoned essay with good, solid points.

My only quibble is that Charm++ is not "a framework for particle simulation methods". While the molecular dynamics program NAMD has been using it for 20 years, which is why I know of Charm++, it wasn't designed specifically for particle simulation methods, nor is restricted to that topic. Quoting from http://charm.cs.illinois.edu/newPapers/08-09/paper.pdf :

> NAMD, from our oldest collaboration, is a program for biomolecular modeling [2]; OpenAtom is a Car-Parinello MD program used for simulation of electronic structure (in nanomaterials, as well as biophysics) [3]; ChaNGa, an astronomy code [13]; and RocStar, a code for simulating solid-propellant rockets, such as those in the space shuttle solid rocket booster

Yes, "high performance computing" is dying. There's no commercial market for it. Check the list of the top 500 supercomputers in the world.[1] The top 10 are all Government operations. In the top 25, there are a few oil companies, mostly running big arrays of Intel Xeons.

CPU clock speeds maxed out between 3-4GHz a decade ago. Nobody develops special supercomputing CPUs any more. The market is tiny. Old supercomputer guys reminisce about the glory days when IBM, Cray, Control Data, and UNIVAC devoted their best R&D efforts to supercomputers. That ended 30 years ago.

Supercomputers have poor price-performance. Grosch's Law [2] stopped working a long time ago. Maximum price/performance today is achieved with racks of midrange CPUs, which is why that's what every commercial data center has. Now everybody has to deal with clusters of machines. So cluster interconnection has become mainstream, not the province of supercomputing.

[1] http://www.top500.org/list/2014/11/ [2] http://en.wikipedia.org/wiki/Grosch%27s_law

I've heard there is a new effort being led by Torsten Hoefler to modernize MPI and address a number of the issues mentioned in this article.

http://htor.inf.ethz.ch/

I was at a talk of his last year and there are a number of fault tolerant MP algorithms being drawn in. MPI hasn't been updated in ages, I don't think that necessarily means we need to ditch it, it just means the standard needs to be modernized. I don't feel very strongly about this since working with MPI is a huge pain in the ass and it seems like the challenge of modernizing it is just gargantuan.

Also I'm not familiar with spark, but isn't Chapel a decade old at this point and barely works at all? I tried their compiler last summer and it took 5 minutes to compile hello world, hopefully its improving.

I spent a few months learning (modern) Fortran a year or two ago. My chief obstacle was the difficulty involved in finding modern tutorials. I don't want to have to read tutorials written in 1994 whose focus is getting people used to F77 up to speed. I've yet to find a tutorial that approached teaching F08 as if it was a new language, which is what I feel is needed.

Even in F08 there's a lot of backwards-compatibility cruft still left in the language, too. The IO model still provides very little abstraction and is based on tape drives. You can/have to "rewind" files. There are obscure "unit descriptors" that manifest themselves as integer literals in most code posted online which makes it a chore to learn from. As far as I can tell there is no functionality that approximates the behaviour of C++'s streams.

It's fast as hell, and the GNU compiler is mature and well-developed, but Fortran remains a horrid language for doing any sort of interactive programming. It's best used if you just give it some arguments, let it run free, and then have it return some object or value that a more sane language can then interpret and present to the user for a decision.

There is little reason to learn a language where the only sane choice for doing input/output involves calling your Fortran module from a python script and letting the python handle i/o.

The article is somewhat confusingly written (or maybe I was just confused by "killing it" colloquially meaning "doing well"), but even a cursory glance shows that it is not arguing that non-distributed high performance computing is relevant. I'm not sure what your post is responding to.

It is instead arguing that traditional HPC is being made irrelevant because traditional HPC uses MPI (the first successful distributed/parallel computing library), which is increasingly irrelevant in favor of newer libraries for the same task.