Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reflections on 30 years of HPC programming

chapel-lang.org

Post reply on HN
Print view View original

81–90 of 129 posts

Re: Reflections on 30 years of HPC programming

#81

Post by [deleted] »

suuuuuuuu wrote: post #64
pjmlp wrote: post #57

Earlier quoted context omitted.

Says those that don't know CUDA. You can program CUDA in standard C++20, with CUDA libraries hidding the language extensions. I love when C and C++ dialects are C and C++ when it matters, and not when it doesn't help to sell the ideas being portrayed.

Sorry, I wasn't aware of these developments (having abandoned CUDA for hardware-agnostic solutions before 2020). It doesn't change my point anyway, if it's specific to a single vendor. I'm extremely dubious that such an opaque abstraction can actually solve the (true) problem. "Not having to write CUDA" is not enough - how do you tune performance? Parallelization strategies, memory prefetching and arrangement in on-c…

[deleted]

Re: Reflections on 30 years of HPC programming

#82

Post by guywithahat »

I like the idea of chapel, but I'm not sure I agree with a lot of their design choices. Some of the parallelization features seem like they just copied OpenMP without meaningfully improving on it. They also kept exceptions, which are generally on their way out, especially in compiled languages (Go, Rust, Zig, and while they exist in modern C++ they are introducing more ways to not use them). I think a new HPC language is possible but I'm not sure this is the one

Re: Reflections on 30 years of HPC programming

#83

Post by bradcray »

SiempreViernes wrote: post #74
pjmlp wrote: post #56

Earlier quoted context omitted.

Not really, you should actually read that section a few times as well. > A fact of life in HPC is that the community has many large, long-lived codes written in languages like Fortran, C, and C++ that remain important. Such codes keep those languages at the forefront of peoples’ minds and sometimes lead to the belief that we can’t adopt new languages. > In large part because of the previous point, our programming not…

The fact that the author is a developer of Chapel pretty neatly explains why "no new language was adopted" is valued as failure, the article itself makes little effort to argue for that value judgment.

Author here: I didn't go into more detail on this than https://chapel-lang.org/blog/posts/30years/#maybe-hpc-doesnt... because I felt like the article was long enough already and that I'd recently covered that topic in detail in this series https://chapel-lang.org/blog/series/10-myths-about-scalable-... summarized here https://chapel-lang.org/blog/posts/10myths-part8/#summary

Re: Reflections on 30 years of HPC programming

#84

Post by bradcray »

RhysU wrote: post #25

> we have failed to broadly adopt any new compiled programming languages for HPC The article neglects that all of C, C++, and Fortran have evolved over the last 30 years. Also, you'll find significant advances in the HPC library ecosystem over the trailing years. Consider, for example, Trilinos ( https://trilinos.github.io/index.html ) or Dakota ( https://dakota.sandia.gov/about-dakota/ ) both of which push a ton of…

The evolution of C, C++, and Fortran is touched on in a sidebar, although admittedly very briefly:

> Champions of Fortran, C++, MPI, or other entries on this list could argue that…

Re: Reflections on 30 years of HPC programming

#85

Post by kinow »

Xcelerate wrote: post #33

I wonder how much of the programming language problem is due to churn of the user base. Looking over many comments in this thread, I see “Oh, back when I did HPC...” I used Titan for my own work back in 2012. But after my PhD, I never touched HPC again. So the people writing the code use what’s there but don’t stay long enough to help incentivize new or better languages. Now on the hardware side (e.g., design of inte…

Churn of the user base could be playing a role in this, but I think it may not be too significant. In Europe there are multiple universities with HPC masters, which provide new users/devs to HPC. I worked with HPCs in New Zealand, and now I am doing the same in Spain. We hire multiple people from other HPC centers in Germany/UK/Italy, and equally lose people to those sites.

I think the field is actually increasing with AI, digital twins, more industry projects (CFD, oil, models for fisheries, simulations for health diseases, etc.).

Re: Reflections on 30 years of HPC programming

#86

Post by adrian_b »

fcanesin wrote: post #31
suuuuuuuu wrote: post #28

Earlier quoted context omitted.

If you think C++ is the best here, then I don't think you've actually worked in this space nor appreciated the actual problems these languages try to solve. In particular because you can't program accelerators with C++. Memory bandwidth is often the problem, yes. Language abstractions for performance aim to, e.g., automatically manage caches (that must be handled manually in performant GPU code, for instance) with op…

Wait what!? I have been programming CUDA since 2009 and specifically remember it being pushed to C++ as main development language for the first few years, after a brief "CUDA C extension" period.

CUDA variants extend several programming languages, including C, C++ and Fortran.

None of the extended languages is the same as the same as the base language, in the same way like OpenMP C++ in not the same as C++ or OpenMP Fortran is not the same as Fortran or SYCL is not the same as C++.

The extended languages include both extensions and restrictions of the base language. In the part of a program that will run on a GPU you can do things that cannot be done in the base language, but there also parts of the base language, e.g. of C++, which are forbidden.

All these extended languages have the advantage that you can write in a single source file a complete multithreaded program, which has parts running concurrently on a CPU and part running concurrently on a GPU, but for the best results you must know the rules that apply to the language accepted by each of them. It is possible to write program that run without modification on either a CPU or a GPU, but this is paid by a lower performance on any of them, because such a program uses only generic language features that work on any of them, instead of taking advantage of specific features.

Re: Reflections on 30 years of HPC programming

#87

Post by Joel_Mckay »

RandomTeaParty wrote: post #79
Joel_Mckay wrote: post #20

Earlier quoted context omitted.

People did try to create an OTP in HDL at one point. And Erlang has already run many telecom infrastructures for decades. Surprising given how fragile the multi-host implementation has proven. Erlang/Elixir are neat languages, and right next to Julia for fun. =3

Erlang is about reliability, hpc is about performance (literally in the name)

Most HPC job queue cluster partition batching I saw was stone-age primitive by comparison. =3

Re: Reflections on 30 years of HPC programming

#88

Post by GuB-42 »

j4k0bfr wrote: post #9
jandrewrogers wrote: post #3

I can easily explain this, having worked in this space. The new languages don’t actually solve any urgent problems. How people imagine scalable parallelism works and how it actually works doesn’t have a lot of overlap. The code is often boringly single-threaded because that is optimal for performance. The single biggest resource limit in most HPC code is memory bandwidth. If you are not addressing this then you are n…

I'm pretty interested in realtime computing and didn't realise C++ was considered bandwidth efficient! Coming from C, I find myself avoiding most 'new' C++ features because I can't easily figure out how they allocate without grabbing a memory profiler.

C++ is like C with extra features, but you don't need to use them.

If you want control over your memory, you can do pointers the C way, but you still have features like templates, namespaces, etc... Another advantage of C++ is that it can go both high and low level within the same language.

Disadvantage of C++ is mostly related to portability and interop. Things like name mangling, constructors, etc... can be a problem. Also, C++ officially doesn't support some C features like "restrict". In practice, you often can use them, but it is nonstandard. Probably not a concern for HPC.

Re: Reflections on 30 years of HPC programming

#89

Post by jeffreygoesto »

Joel_Mckay wrote: post #49
j4k0bfr wrote: post #48

Earlier quoted context omitted.

I'm talking almost exactly about this haha. Flight software representation!! Although I have no experience programming FPGAs, I hope to gain some soon. They seem like the ultimate solution to my IO woes.

A bit legacy these days, but I liked the old Zynq 7020 dual core ARM with reasonable LUT counts. https://www.youtube.com/watch?v=FujoiUMhRdQ https://github.com/Spyros-2501/Z-turn-Board-V2-Diary https://www.youtube.com/@TheDevelopmentChannel/playlists https://myirtech.com/list.asp?id=708 The Debian Linux example includes how to interface hardware ports. Almost always better to go with a simpler mcu if one can get away…

That chip was hitting a sweet spot in terms of DRAM controller and distributing memory bandwidth between CPU cores and fabric. Xilinx was very afraid of screwing this up and running into bottlenecks. One of the best balanced chips in that regard with a great controller. Your best bet still was to keep everything in blockram as much as possible and only read and write DRAM once at every end of the computation...

Re: Reflections on 30 years of HPC programming

#90

Post by iamcreasy »

jandrewrogers wrote: post #3

I can easily explain this, having worked in this space. The new languages don’t actually solve any urgent problems. How people imagine scalable parallelism works and how it actually works doesn’t have a lot of overlap. The code is often boringly single-threaded because that is optimal for performance. The single biggest resource limit in most HPC code is memory bandwidth. If you are not addressing this then you are n…

Julia language is also used for HPC according to their webpage citing performance parity with C++. Would it be correct to infer Julia also provides the same level of memory bandwidth control?
Post reply on HN