W-why are they using that picture at the start when their source is an essay on exactly the way they're misusing it?
HPC is dying, and MPI is killing it
51–60 of 125 posts
Re: HPC is dying, and MPI is killing it
#52I used MPI (Message Passing Interface...) back in the day.
But it was a pain, especially since our code was a mix of c (which was easy to mpi) and ada (not so easy). Its pretty low level stuff (I think we used Open MPI). All the nodes need to have MPI set up and configured, fine if you have a team willing to do it but these days....
mpiexec -n 10 myprocess
I think we liked it because the processes would be put to sleep by the mpi daemon until a message arrived. You can sleep and wait for a message with sockets now I think. Its been a while since I've used the unix IPC (Interprocess communication).
I don't think I'll miss it.
Re: HPC is dying, and MPI is killing it
#53Earlier quoted context omitted.
Did you RTFA? It's about MPI: "MPI is a language-independent communications protocol used to program parallel computers." Runs fine on commodity clusters.
What would you define as a "commodity cluster"? To me it's a 512-core vendor-specific blade server with special interfaces to get more bandwith at lower latency across longer links. But maybe i'm just an old fogey.
1) Not vendor specific
2) Not blades
3) 10Gbe, not special interfaces
That is a commodity cluster.
Re: HPC is dying, and MPI is killing it
#54Earlier quoted context omitted.
Erlang itself is not good for the type of numerical computation typical done in HPC. It is amazing as a backend, and there are several ways it can call code written in other languages through the use of ports or NIFS, but if you try to do massive number-crunching using its own libraries then you're going to be unhappy with the results.
How about in simulations where a large number of cellular automata are interacting with each other, but individually only carrying out simple computations?
You cannot make an efficient fluid mechanics simulation on a 4000x4000x4000 grid if you set up a separate process for each individual gridcell. More efficient to just store your numbers in 3d arrays.
Re: HPC is dying, and MPI is killing it
#55in High Performance Computing, there is (1) 3-dimensional simulations (weather, fluid dynamics, structural mechanics, all kinds of physics simulations, like magnetic storms in space or nuclear reactors etc.) and then there is (2) everything else, like data mining, machine learning, genomics etc. Some of the sparse matrix computations in structural mechanics and in some machine learning algorithms have some overlap. B…
They are doing it (interesting things) for lower capex and lower development costs. On opex, good for operations, bad for power consumption (relatively). In terms of absolute performance HPC is absolutely faster. In terms of bangs for bucks, Big Data is hands down faster. Also in terms of accessibility Big Data is hugely easier - I can build you a 100 core big data system for $300k
that's silly: HPC has been pinching pennies before big data was a thing. and the computer industry is biz: you get what you pay for. if you can live with Gb performance, you can drop around $2k (IB card, cables, switches) off your price. But it's not as if the hardware is any different, faster or more accessible.
Re: HPC is dying, and MPI is killing it
#56I am trying to get into distributed computing so this article is particularly interesting to me. I may be mistaken so please excuse my naivety if my points are off marks. I thought MPI was mainly geared towards communication-heavy tasks where the underlying network is specialized, for example infiniband or bus between CPUs. One use of MPI is to manage distributed memory tasks between different physical CPUs while thr…
RDMA doesn't really provide a flat memory model - all it's really doing is minimizing copies when you send a message. more like "put this 100K string into that node at ".
Re: HPC is dying, and MPI is killing it
#57in High Performance Computing, there is (1) 3-dimensional simulations (weather, fluid dynamics, structural mechanics, all kinds of physics simulations, like magnetic storms in space or nuclear reactors etc.) and then there is (2) everything else, like data mining, machine learning, genomics etc. Some of the sparse matrix computations in structural mechanics and in some machine learning algorithms have some overlap. B…
They are doing it (interesting things) for lower capex and lower development costs. On opex, good for operations, bad for power consumption (relatively). In terms of absolute performance HPC is absolutely faster. In terms of bangs for bucks, Big Data is hands down faster. Also in terms of accessibility Big Data is hugely easier - I can build you a 100 core big data system for $300k
But your Big Data system is only good for Big Data. If I want to run a weather prediction model, it is not going to help.
My point is, the big data and the physics simulation people probably do not have a lot of common interests - besides using large amounts of computing power.
Re: HPC is dying, and MPI is killing it
#58MPI is Message Passing Interface, for those who don't know. Author never explicitly says what MPI stands for. https://en.wikipedia.org/wiki/Message_Passing_Interface
Yeah, I had to go back to google after failing to find it defined anywhere on the page. If you don't use something regularly yourself, you may remember the concept but forget the acronym. For want of a couple of sentences of context at the outset, the rest of the article was quite inaccessible. Maybe there's a lesson for the author here.
Re: HPC is dying, and MPI is killing it
#59Earlier quoted context omitted.
Again I'm not sure if I agree or disagree with this. My hatred of MPI is only outweighed by the fact that I can use it... and my code works. I think a large part of the inertia behind MPI is legacy code. Often the most complex part of HPC scientific codes is the parallel portion and the abstractions required to perform them (halo decomposition etc). I can't imagine there are too many grad students out there who are e…
Oh, sure. I don't think anyone should start rewriting old codes; but as new projects start, I think we have a lot more options out there than we did 10 years ago, and it's worth looking closely at them before starting, rather than defaulting to something. Especially since, once you start, you're probably pretty much locked into whatever you chose for a decade or so.
So, say you wanted to write a weather model, or engineering fluid mechanics model. Which options (besides MPI) you would look at?
Re: HPC is dying, and MPI is killing it
#60Earlier quoted context omitted.
Did you RTFA? It's about MPI: "MPI is a language-independent communications protocol used to program parallel computers." Runs fine on commodity clusters.
What would you define as a "commodity cluster"? To me it's a 512-core vendor-specific blade server with special interfaces to get more bandwith at lower latency across longer links. But maybe i'm just an old fogey.
blades were never more than a marketing trick: the offer nothing that can't be achieved in a standard chassis. there were a few multi-chassis SMP/NUMA machines that had cache coherency over external interfaces, but that was neither commodity nor HPC.