HPC is dying, and MPI is killing it • Hacker News Board

Re: HPC is dying, and MPI is killing it

#11

Post by duskwuff » Tue, Apr 07, 2015, 7:19 PM UTC

na85 wrote: post #8

shepardrtc wrote: post #5

The Fortran Standards Committee is attempting to make HPC easier through the use of coarrays, which are essentially massive abstractions over MPI. I really wish people would give Fortran a second chance. It has come a long way from the ancient, all-caps days.

I spent a few months learning (modern) Fortran a year or two ago. My chief obstacle was the difficulty involved in finding modern tutorials. I don't want to have to read tutorials written in 1994 whose focus is getting people used to F77 up to speed. I've yet to find a tutorial that approached teaching F08 as if it was a new language, which is what I feel is needed. Even in F08 there's a lot of backwards-compatibilit…

> You can/have to "rewind" files.

This isn't necessarily a Fortran-specific thing. The standard C library includes a rewind(fd) function, equivalent to lseek(fd, 0, SEEK_SET).

Re: HPC is dying, and MPI is killing it

#12

Post by acadien » Tue, Apr 07, 2015, 7:20 PM UTC

tormeh wrote: post #9

acadien wrote: post #6

I've heard there is a new effort being led by Torsten Hoefler to modernize MPI and address a number of the issues mentioned in this article. http://htor.inf.ethz.ch/ I was at a talk of his last year and there are a number of fault tolerant MP algorithms being drawn in. MPI hasn't been updated in ages, I don't think that necessarily means we need to ditch it, it just means the standard needs to be modernized. I don't…

A decade is no age for a language, though. Creating a language with compiler can be done very quickly, but creating a good language with a good compiler and a good standard library takes time. And then it needs to catch on. This requires about a decade++. Scala is 12 years old, Go is 6 years old and Clojure is 8 years old.

Yeah good point. I just felt it might be a misleading of the author to suggest Chapel as an alternative when you cannot possibly write a useful program with it.

Re: HPC is dying, and MPI is killing it

#13

Post by ljdursi » Tue, Apr 07, 2015, 7:23 PM UTC

acadien wrote: post #6

I've heard there is a new effort being led by Torsten Hoefler to modernize MPI and address a number of the issues mentioned in this article. http://htor.inf.ethz.ch/ I was at a talk of his last year and there are a number of fault tolerant MP algorithms being drawn in. MPI hasn't been updated in ages, I don't think that necessarily means we need to ditch it, it just means the standard needs to be modernized. I don't…

Certainly Prof Hoefler has done a lot of work driving the design of updated remote-memory access for of MPI-3, and any further progress would be welcomed; but I don't think any modernizing of MPI can fix the basic problem. At the end of the day, it's just too low level for application developers, while being too high-level for tool developers. There are parts of MPI which don't share this problem so much - the collective operations, and especially MPI-IO; but the disconnect between what people need to build either tools or scientific applications and what MPI provides just seems too great.

For Chapel, it depends on what you count; it very heavily borrows from ZPL, which is much older, but Chapel itself was only released in 2009. It is already competitive with MPI in performance in simple cases, while operating at a much higher level of abstraction. Whether Chapel, or Spark, are the right answers in the long term, I don't know; but there's a tonne of other options out there that are worth exploring.

Re: HPC is dying, and MPI is killing it

#14

Post by tormeh » Tue, Apr 07, 2015, 7:26 PM UTC

shepardrtc wrote: post #5

The Fortran Standards Committee is attempting to make HPC easier through the use of coarrays, which are essentially massive abstractions over MPI. I really wish people would give Fortran a second chance. It has come a long way from the ancient, all-caps days.

I LIKE PROGRAMMING IN ALL-CAPS. IT MAKES EVERYTHING I WRITE LOOK OFFICIAL, NO-NONSENSE AND GENERALLY HARD-CORE. INSTEAD OF HAVING A CONVERSATION WITH THE COMPILER I INSTEAD SHOUT!!! COMMANDS AT IT.

Re: HPC is dying, and MPI is killing it

#15

Post by ljdursi » Tue, Apr 07, 2015, 7:27 PM UTC

shepardrtc wrote: post #5

The Fortran Standards Committee is attempting to make HPC easier through the use of coarrays, which are essentially massive abstractions over MPI. I really wish people would give Fortran a second chance. It has come a long way from the ancient, all-caps days.

I think coarrays have a lot of promise; it's less ambitious than the single unified picture of a program that Chapel or UPC has, but maybe that's a feature, not a bug, for incrementally changing how we do things. One problem has been that implementations of coarrays depended on MPI-2 features which were brittle and not super optimized (because they weren't widely used); but with MPI-3s RMA, or gasnet (which I think gfortran is starting to use?) it could start being more practical for production use.

Re: HPC is dying, and MPI is killing it

#16

Post by acadien » Tue, Apr 07, 2015, 7:28 PM UTC

ljdursi wrote: post #13

acadien wrote: post #6

I've heard there is a new effort being led by Torsten Hoefler to modernize MPI and address a number of the issues mentioned in this article. http://htor.inf.ethz.ch/ I was at a talk of his last year and there are a number of fault tolerant MP algorithms being drawn in. MPI hasn't been updated in ages, I don't think that necessarily means we need to ditch it, it just means the standard needs to be modernized. I don't…

Certainly Prof Hoefler has done a lot of work driving the design of updated remote-memory access for of MPI-3, and any further progress would be welcomed; but I don't think any modernizing of MPI can fix the basic problem. At the end of the day, it's just too low level for application developers, while being too high-level for tool developers. There are parts of MPI which don't share this problem so much - the collec…

Again I'm not sure if I agree or disagree with this. My hatred of MPI is only outweighed by the fact that I can use it... and my code works.

I think a large part of the inertia behind MPI is legacy code. Often the most complex part of HPC scientific codes is the parallel portion and the abstractions required to perform them (halo decomposition etc). I can't imagine there are too many grad students out there who are eager to re-write a scientific code in a new language that is unproven and requires developing a skill set that is not yet useful in industry (who in industry has ever heard of Chapel or Spark??). Not to mention that re-writing legacy codes means you're delaying from getting results. Its just a terrible situation to be in.

Re: HPC is dying, and MPI is killing it

#17

Post by gaius » Tue, Apr 07, 2015, 7:30 PM UTC

Animats wrote: post #4

Yes, "high performance computing" is dying. There's no commercial market for it. Check the list of the top 500 supercomputers in the world.[1] The top 10 are all Government operations. In the top 25, there are a few oil companies, mostly running big arrays of Intel Xeons. CPU clock speeds maxed out between 3-4GHz a decade ago. Nobody develops special supercomputing CPUs any more. The market is tiny. Old supercomputer…

CPUs any more

CPUs, err, schmee-PUs. It's all about the interconnect and people can and do make special interconnects.

Re: HPC is dying, and MPI is killing it

#18

Post by gaius » Tue, Apr 07, 2015, 7:31 PM UTC

shepardrtc wrote: post #5

The Fortran Standards Committee is attempting to make HPC easier through the use of coarrays, which are essentially massive abstractions over MPI. I really wish people would give Fortran a second chance. It has come a long way from the ancient, all-caps days.

As the old saying goes, a good developer can write FORTRAN in any language.

Re: HPC is dying, and MPI is killing it

#19

Post by ljdursi » Tue, Apr 07, 2015, 7:32 PM UTC

dalke wrote: post #2

I came into the essay with suspicion. A map-reduce system like Hadoop isn't a good fit for HPC problems, and I thought it would argue that MPI is old => it's stuck in the past. Instead, and to my joy, it was a well-reasoned essay with good, solid points. My only quibble is that Charm++ is not "a framework for particle simulation methods". While the molecular dynamics program NAMD has been using it for 20 years, which…

Thanks! I'm probably mischaracterizing Charm++ a bit, because I'm most familiar with it in particle context (OpenAtom, ChaNGa, NAMD). I guess it's probably particularly used in that context just because it's so good with very fine-grained distributions of work units. I'll edit that line in the article.

Re: HPC is dying, and MPI is killing it

#20

Post by ljdursi » Tue, Apr 07, 2015, 7:34 PM UTC

acadien wrote: post #16

ljdursi wrote: post #13

Earlier quoted context omitted.

Certainly Prof Hoefler has done a lot of work driving the design of updated remote-memory access for of MPI-3, and any further progress would be welcomed; but I don't think any modernizing of MPI can fix the basic problem. At the end of the day, it's just too low level for application developers, while being too high-level for tool developers. There are parts of MPI which don't share this problem so much - the collec…

Again I'm not sure if I agree or disagree with this. My hatred of MPI is only outweighed by the fact that I can use it... and my code works. I think a large part of the inertia behind MPI is legacy code. Often the most complex part of HPC scientific codes is the parallel portion and the abstractions required to perform them (halo decomposition etc). I can't imagine there are too many grad students out there who are e…

Oh, sure. I don't think anyone should start rewriting old codes; but as new projects start, I think we have a lot more options out there than we did 10 years ago, and it's worth looking closely at them before starting, rather than defaulting to something. Especially since, once you start, you're probably pretty much locked into whatever you chose for a decade or so.