Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

HPC is dying, and MPI is killing it

dursi.ca

Post reply on HN
Print view View original

81–90 of 125 posts

Re: HPC is dying, and MPI is killing it

#81

Post by darkmighty »

acadien wrote: post #22
gaius wrote: post #17

Earlier quoted context omitted.

CPUs any more CPUs, err, schmee-PUs. It's all about the interconnect and people can and do make special interconnects.

Yeah the 5 dimensional torus network used at MIRA is just too cool not to bring up here. Modern supercomputers are increasingly become less and less discrete. https://computing.llnl.gov/tutorials/bgq/

What's a 5 dimensional torus? I know a 3D is Circle x Circle, is a 5-d torus a Circle x Circle x Circle x Circle?

If so, that could be interpreted as simply a 4D square grid which wraps around the edges, right? (just as a 3D torus is a 2D grid which wraps)

Re: HPC is dying, and MPI is killing it

#82

Post by dalke »

threeseed wrote: post #75
dalke wrote: post #2

I came into the essay with suspicion. A map-reduce system like Hadoop isn't a good fit for HPC problems, and I thought it would argue that MPI is old => it's stuck in the past. Instead, and to my joy, it was a well-reasoned essay with good, solid points. My only quibble is that Charm++ is not "a framework for particle simulation methods". While the molecular dynamics program NAMD has been using it for 20 years, which…

People need to be careful about talking about Hadoop as just a map-reduce system. It's YARN container system is flexible enough to run any JVM application. For example we use it to run an autoscaling ElasticSearch cluster alongside our Hadoop workloads. And we are actively investigating using it to run our Scala microservices.

Even years after Amazon started selling a lot more than just books, if people were asked "what does Amazon sell?", the answer was often "books."

I looked at YARN now. I've not heard of it before. It doesn't look like it has anything to do with the topic at hand. How would one build an explicit solver for a 1D diffusion equation, corresponding to the examples given in the "HPC is dying, ..." article, using YARN?

How do you do checkpointing so you can restart your 10 million atom simulation should there be a system fault after 2 weeks of run-time? (Checkpoints need about 220 MB; each atom has an x,y,z position as well as a vx,vy,vz velocity vector. Also, it needs to be at the same timestep across the entire distributed machine.)

Instead, it looks like YARN is designed for service-based components, where the components are relatively independent from each other, and where failure recovery is mostly a matter of starting a new service and resending the request.

If my understanding is correct, then it's certainly more capable than map-reduce. But not in a direction that's relevant for most current HPC.

Re: HPC is dying, and MPI is killing it

#83

Post by dalke »

yellowapple wrote: post #77
Animats wrote: post #4

Yes, "high performance computing" is dying. There's no commercial market for it. Check the list of the top 500 supercomputers in the world.[1] The top 10 are all Government operations. In the top 25, there are a few oil companies, mostly running big arrays of Intel Xeons. CPU clock speeds maxed out between 3-4GHz a decade ago. Nobody develops special supercomputing CPUs any more. The market is tiny. Old supercomputer…

> CPU clock speeds maxed out between 3-4GHz a decade ago. Even for x86 this isn't true (4+GHz is at least possible), let alone platforms like POWER which have already pushed beyond 5GHz. Fancier things like vacuum-channel transistors, graphene transistors, etc. could push that even further once they break into commercial viability. Not that clock speed alone really matters all that much compared to the other performa…

"Today I learned that Blue Gene was a figment of my imagination" .. and I learned that Anton doesn't exist either.

Re: HPC is dying, and MPI is killing it

#84

Post by dalke »

dorfsmay wrote: post #46
east2west wrote: post #39

I am trying to get into distributed computing so this article is particularly interesting to me. I may be mistaken so please excuse my naivety if my points are off marks. I thought MPI was mainly geared towards communication-heavy tasks where the underlying network is specialized, for example infiniband or bus between CPUs. One use of MPI is to manage distributed memory tasks between different physical CPUs while thr…

> I thought MPI was mainly geared towards communication-heavy tasks where the underlying network is specialized The beauty of mpi is: * its definition is completely open * it segregate the high level message passing interface from the low level stuff This means that code that was written on cheap old commodity network gear over tcp/ip will work on brand new specialised hardware using their own protocol. Because it's…

I think you've omitted something. PVM had that same beauty, but MPI won over PVM. Do you have any idea why?

Re: HPC is dying, and MPI is killing it

#85

Post by acadien »

darkmighty wrote: post #81
acadien wrote: post #22

Earlier quoted context omitted.

Yeah the 5 dimensional torus network used at MIRA is just too cool not to bring up here. Modern supercomputers are increasingly become less and less discrete. https://computing.llnl.gov/tutorials/bgq/

What's a 5 dimensional torus? I know a 3D is Circle x Circle, is a 5-d torus a Circle x Circle x Circle x Circle? If so, that could be interpreted as simply a 4D square grid which wraps around the edges, right? (just as a 3D torus is a 2D grid which wraps)

Think of it as loops in 5 dimensions (x,y,z,a,b). I believe each node connects to 10 different neighboring nodes (2 in each of 5 dimensions), although I could be mistaken on that. You can actually tune which direction you prefer the nodes to communicate over by passing certain flags when you submit a job.

Also here's an image... which I admit is not terribly useful, but its what the national lab people put out. https://computing.llnl.gov/tutorials/bgq/images/5Dtorus.400p...

Re: HPC is dying, and MPI is killing it

#86

Post by acadien »

sampo wrote: post #57
sgt101 wrote: post #47

Earlier quoted context omitted.

They are doing it (interesting things) for lower capex and lower development costs. On opex, good for operations, bad for power consumption (relatively). In terms of absolute performance HPC is absolutely faster. In terms of bangs for bucks, Big Data is hands down faster. Also in terms of accessibility Big Data is hugely easier - I can build you a 100 core big data system for $300k

But your Big Data system is only good for Big Data. If I want to run a weather prediction model, it is not going to help. My point is, the big data and the physics simulation people probably do not have a lot of common interests - besides using large amounts of computing power.

There are big data people running on supercomputers too. I know there are people writing custom asynch job managers to handle big data type problems because the top supercomputers have low memory latency.

Also I think the dichotomy you're looking for is IO bound vs CPU bound problems. Although certainly there are a plethora of different kinds of IO bound problems (asynch vs synch or disk bound vs memory bound vs cache bound).

Re: HPC is dying, and MPI is killing it

#87

Post by shepardrtc »

rz2k wrote: post #31
shepardrtc wrote: post #24

Earlier quoted context omitted.

Erlang itself is not good for the type of numerical computation typical done in HPC. It is amazing as a backend, and there are several ways it can call code written in other languages through the use of ports or NIFS, but if you try to do massive number-crunching using its own libraries then you're going to be unhappy with the results.

How about in simulations where a large number of cellular automata are interacting with each other, but individually only carrying out simple computations?

You certainly can do that. In fact, I believe I saw an example of that in a tutorial somewhere talking about what Erlang/Elixir would be good for (I include Elixir because its the same VM controlling processes underneath, but a "nicer" syntax on top). Now, would it be the best language for that? Depends. If you're looking just for speed and massive computations, then no. In the end, C++ pretty much rules everything in that regard (except for maybe Fortran in some instances). But, I would say that it might be more fun to set it up in Erlang/Elixir. And you could pretty easily expand the whole thing just by adding more processors and simply telling it to spawn more processes (the Erlang VM is pretty awesome). I would almost say that an Erlang version of it would feel more life-like. You could probably experiment with it on the fly more easily, too. Kill off a process here or there and see what happens, etc.

Re: HPC is dying, and MPI is killing it

#88

Post by m_mueller »

gaius wrote: post #18
shepardrtc wrote: post #5

The Fortran Standards Committee is attempting to make HPC easier through the use of coarrays, which are essentially massive abstractions over MPI. I really wish people would give Fortran a second chance. It has come a long way from the ancient, all-caps days.

As the old saying goes, a good developer can write FORTRAN in any language.

Writing, sure. Getting it to run as fast -> replace any language with C and add some boilerplate.

Re: HPC is dying, and MPI is killing it

#89

Post by CHY872 »

ljdursi wrote: post #45
east2west wrote: post #39

I am trying to get into distributed computing so this article is particularly interesting to me. I may be mistaken so please excuse my naivety if my points are off marks. I thought MPI was mainly geared towards communication-heavy tasks where the underlying network is specialized, for example infiniband or bus between CPUs. One use of MPI is to manage distributed memory tasks between different physical CPUs while thr…

I agree that languages that rely on tracing GC seem like they're fundamentally at a disadvantage when it comes to pushing the envelope of single-node performance; the best article I've read arguing this was actually in the context of mobile games, rather than HPC, but I can't for the life of me find the article now. I don't know if Spark itself is the right way forward; but it's an example of a very productive high-l…

I'd be very wary of criticising Spark based on the technologies it's built on.

In particular, a decent garbage collector will give you performance dependent on the number of live objects (typically low) and not on the number of allocations and deallocations, as you might see in a non-garbage collected language. This gives great allocation performance and reduces overheads.

The disadvantages can be (potentially long) GC pauses and higher overall memory requirements, but in practice this isn't usually a problem for non-interactive systems.

Of course, if you do have a device with low memory and low tolerance for GC pauses (i.e. mobile gaming) there might be a problem.

The main disadvantage seems to be less predictable performance; which could be a problem in domains which require good timing performance, but that's not really Spark's problem.

A GC'd language is also generally easier to program in, since one doesn't have to (in general) worry about memory management, so it's generally a lot easier to program very large systems with lots of moving parts.

Re: HPC is dying, and MPI is killing it

#90

Post by WallWextra »

Animats wrote: post #4

Yes, "high performance computing" is dying. There's no commercial market for it. Check the list of the top 500 supercomputers in the world.[1] The top 10 are all Government operations. In the top 25, there are a few oil companies, mostly running big arrays of Intel Xeons. CPU clock speeds maxed out between 3-4GHz a decade ago. Nobody develops special supercomputing CPUs any more. The market is tiny. Old supercomputer…

NEC is still making their weird vector computers. I have no idea who buys them, but there's a hotchips about it.
Post reply on HN