61–70 of 125 posts
I built a tiny cluster in my basement (a prototype) and looked at MPI and decided that it was way to complicated, so I just built something that pushes the essential bits, pretty much without abstraction, to the nodes and was done. The cluster is a very specific solution, so I felt justified in not looking at MPI. And now the decision feels even more justified.
Too bad he didn't talk about GPGPU killing MPI too or not. I don't know enough to say. I'm not familiar with the HPC space but I thought a lot of new work, at least in machine learning, was migrating to GPGPU instead of traditional CPUs. The compute per $ or per watt payoff is too large to ignore.
Earlier quoted context omitted.
A decade is no age for a language, though. Creating a language with compiler can be done very quickly, but creating a good language with a good compiler and a good standard library takes time. And then it needs to catch on. This requires about a decade++. Scala is 12 years old, Go is 6 years old and Clojure is 8 years old.
Yeah good point. I just felt it might be a misleading of the author to suggest Chapel as an alternative when you cannot possibly write a useful program with it.
Chapel may not be an appropriate replacement for all MPI programs, but it can be used for some programs today.
This happened with Smalltalk. I was a Smalltalk coder. I thought it was the best thing since sliced bread. It has always been clear to me that Smalltalk is far superior to Java. I left the company after a little while, to do C++ graphics. I later heard that my former employer rewrote their Smalltalk application in Java. Now no one uses Smalltalk anymore. While Objective-C is based on Smalltalk, Smalltalk was far easi…
I think that Smalltalk has the right level of abstraction and a lot of very good other things about it.
MPI was, as the article points out, the wrong abstraction for the problem. If MPI dies, I am ok with that.
I am sad that Smalltalk is not more widely used.
The Fortran Standards Committee is attempting to make HPC easier through the use of coarrays, which are essentially massive abstractions over MPI. I really wish people would give Fortran a second chance. It has come a long way from the ancient, all-caps days.
I spent a few months learning (modern) Fortran a year or two ago. My chief obstacle was the difficulty involved in finding modern tutorials. I don't want to have to read tutorials written in 1994 whose focus is getting people used to F77 up to speed. I've yet to find a tutorial that approached teaching F08 as if it was a new language, which is what I feel is needed. Even in F08 there's a lot of backwards-compatibilit…
This happened with Smalltalk. I was a Smalltalk coder. I thought it was the best thing since sliced bread. It has always been clear to me that Smalltalk is far superior to Java. I left the company after a little while, to do C++ graphics. I later heard that my former employer rewrote their Smalltalk application in Java. Now no one uses Smalltalk anymore. While Objective-C is based on Smalltalk, Smalltalk was far easi…
What let Java get ahead of Smalltalk for me personally, as someone getting into programming in 1996, was that i could write it in the text editor i already had, compile it with a compiler i could get for free, and then post the source code on Geocities (actually, Xoom - remember that?) to share with others.
Whereas when i tried to get into Smalltalk, the first thing i had to do was learn my way around this wacky environment with its strange class browser and ultra-retro window manager, and get my head around the fact that my source code wasn't anywhere particular, and yet was everywhere, and that if i wanted to share your code, i had to somehow "file out", and then hope that my internet friends could successfully "file in" to their own potentially modified images. Once i'd got hold of the tools at all, that is.
Which is not to say that the Smalltalk environment was not better than Notepad/DOS box/javac, because of course it was. It just didn't lend itself to adoption and spread nearly as well. It was a tool for masters, with affordance for apprentices.
Also, Java had pretty good networking right in the standard library, and networking was really exciting in 1996.
I am trying to get into distributed computing so this article is particularly interesting to me. I may be mistaken so please excuse my naivety if my points are off marks. I thought MPI was mainly geared towards communication-heavy tasks where the underlying network is specialized, for example infiniband or bus between CPUs. One use of MPI is to manage distributed memory tasks between different physical CPUs while thr…
I agree that languages that rely on tracing GC seem like they're fundamentally at a disadvantage when it comes to pushing the envelope of single-node performance; the best article I've read arguing this was actually in the context of mobile games, rather than HPC, but I can't for the life of me find the article now. I don't know if Spark itself is the right way forward; but it's an example of a very productive high-l…
BTW, thanks for a thought provoking article. You have given me a lot to ponder.
Earlier quoted context omitted.
Did you RTFA? It's about MPI: "MPI is a language-independent communications protocol used to program parallel computers." Runs fine on commodity clusters.
>Runs fine on commodity clusters. Kind of.... For simple, low communication jobs this is true. But when you start trying to find the eigenvectors of a large sparse matrix, communication becomes your bottleneck, at which point MPI on commodity clusters (those without a really fancy interconnect) "works", but not fast enough to be useful.
MPI is Message Passing Interface, for those who don't know. Author never explicitly says what MPI stands for. https://en.wikipedia.org/wiki/Message_Passing_Interface
The capitalization and apposition makes it pretty explicit.
This made me feel old, as I remember the days when we got to learn PVM at distributed computing classes and MPI was presented as something that some people were kind of working on.