Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reflections on 30 years of HPC programming

chapel-lang.org

Post reply on HN
Print view View original

31–40 of 129 posts

Re: Reflections on 30 years of HPC programming

#31

Post by fcanesin »

suuuuuuuu wrote: post #28
jandrewrogers wrote: post #3

I can easily explain this, having worked in this space. The new languages don’t actually solve any urgent problems. How people imagine scalable parallelism works and how it actually works doesn’t have a lot of overlap. The code is often boringly single-threaded because that is optimal for performance. The single biggest resource limit in most HPC code is memory bandwidth. If you are not addressing this then you are n…

If you think C++ is the best here, then I don't think you've actually worked in this space nor appreciated the actual problems these languages try to solve. In particular because you can't program accelerators with C++. Memory bandwidth is often the problem, yes. Language abstractions for performance aim to, e.g., automatically manage caches (that must be handled manually in performant GPU code, for instance) with op…

Wait what!? I have been programming CUDA since 2009 and specifically remember it being pushed to C++ as main development language for the first few years, after a brief "CUDA C extension" period.

Re: Reflections on 30 years of HPC programming

#33

Post by Xcelerate »

I wonder how much of the programming language problem is due to churn of the user base. Looking over many comments in this thread, I see “Oh, back when I did HPC...” I used Titan for my own work back in 2012. But after my PhD, I never touched HPC again. So the people writing the code use what’s there but don’t stay long enough to help incentivize new or better languages. Now on the hardware side (e.g., design of interconnects), that more commonly seems to be a full career.

The other issue is that to really get the value out of these machines, you sort of have to tailor your code to the machine itself to some degree. The DOE likes to fund projects that really show off the unique capabilities of supercomputers, and if your project could in principle be done on the cloud or a university cluster, it’s likely to be rejected at the proposal stage. So it’s sort of “all or nothing” in the sense that many codebases for HPC are one-off or even have machine-specific adaptations (e.g., see LAMMPS). No new general purpose language would really make this easier.

Re: Reflections on 30 years of HPC programming

#34

Post by nnevatie »

Usually a new language is facing the ecosystem mass issue - the previously used language, e.g., C++ has already the critical mass with available libraries and frameworks. Getting to the same level of ecosystem maturity with a new language will take a long time, as seen with Rust.

Re: Reflections on 30 years of HPC programming

#35

Post by pjmlp »

suuuuuuuu wrote: post #28
jandrewrogers wrote: post #3

I can easily explain this, having worked in this space. The new languages don’t actually solve any urgent problems. How people imagine scalable parallelism works and how it actually works doesn’t have a lot of overlap. The code is often boringly single-threaded because that is optimal for performance. The single biggest resource limit in most HPC code is memory bandwidth. If you are not addressing this then you are n…

If you think C++ is the best here, then I don't think you've actually worked in this space nor appreciated the actual problems these languages try to solve. In particular because you can't program accelerators with C++. Memory bandwidth is often the problem, yes. Language abstractions for performance aim to, e.g., automatically manage caches (that must be handled manually in performant GPU code, for instance) with op…

What?!?

NVidia designs CUDA hardware specifically for the C++ memory model, they went through the trouble to refactor their original hardware across several years, so that all new cards would follow this model, even if PTX was designed as polyglot target.

Additionally, ISO C++ papers like senders/receivers are driven by NVidia employees working on CUDA.

Re: Reflections on 30 years of HPC programming

#36

Post by pjmlp »

RhysU wrote: post #25

> we have failed to broadly adopt any new compiled programming languages for HPC The article neglects that all of C, C++, and Fortran have evolved over the last 30 years. Also, you'll find significant advances in the HPC library ecosystem over the trailing years. Consider, for example, Trilinos ( https://trilinos.github.io/index.html ) or Dakota ( https://dakota.sandia.gov/about-dakota/ ) both of which push a ton of…

The authors are aware, as the Chapel compiler makes use of LLVM.

Re: Reflections on 30 years of HPC programming

#37

Post by formerly_proven »

Narishma wrote: post #19
j4k0bfr wrote: post #9

Earlier quoted context omitted.

I'm pretty interested in realtime computing and didn't realise C++ was considered bandwidth efficient! Coming from C, I find myself avoiding most 'new' C++ features because I can't easily figure out how they allocate without grabbing a memory profiler.

I don't think there's much difference between C and C++ (and Rust, etc...) when it comes to this.

Idiomatic/natural rust tends to be a lot heavier on allocations and also physically moving objects around than the other two.

Re: Reflections on 30 years of HPC programming

#39

Post by kmaitreys »

formerly_proven wrote: post #37
Narishma wrote: post #19

Earlier quoted context omitted.

I don't think there's much difference between C and C++ (and Rust, etc...) when it comes to this.

Idiomatic/natural rust tends to be a lot heavier on allocations and also physically moving objects around than the other two.

Can you elaborate on this? Slightly concerned because I have written (and planning to write more) Rust HPC code

Re: Reflections on 30 years of HPC programming

#40

Post by shevy-java »

> Could the reason be that language design is dead, as was asserted by an anonymous reviewer on one of our team’s papers ~30 years ago?

It may not be dead, but it seems much harder for languages to gain adoption.

I think there are several reasons; I also suspect AI contributes a bit to this.

People usually specialize in one or two language, so the more languages exist, the less variety we may see with regards to people ACTUALLY using the language. If I would be, say, 15 years old, I may pick up python and just stick with it rather than experiment and try out many languages. Or perhaps not even write software at all, if AI auto-writes most of it anyway.

Post reply on HN