Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reflections on 30 years of HPC programming

chapel-lang.org

Post reply on HN
Print view View original

121–129 of 129 posts

Re: Reflections on 30 years of HPC programming

#121

Post by Mikhail_K »

crabbone wrote: post #18

As someone who worked for a while and still works in HPC, my impression from this field as compared to eg. programming in finance sector or programming for storage sector is that... HPC is so backwards and far behind, it's really amazing how it's portrayed as some sort of a champion of the field. That's not to say that new things don't happen there, it's just that I find a lot of old stuff that was shown to be bad de…

> you realize they've been written for pre-systemd Linux

So still retaining some kind of sanity and good engineering practices?

Re: Reflections on 30 years of HPC programming

#122

Post by pama »

zozbot234 wrote: post #45
pama wrote: post #30

Earlier quoted context omitted.

HPCs never loved the inefficiencies of anything virtualized (VMs or any containers really), so the shell hacks of module enabled a (limited, but workable) level of reproducibility that was sufficiently composable and usable by researchers who understood the shell. I am not going to defend this tcl hack any further, but I can see how it was the path of least resistance when people tried to stay close to the raw metal…

Containers are an OS sandboxing/namespacing primitive, they don't involve any overhead on their own. The overhead is dependent on what's inside the container besides a single deployed binary.

What you way is true after the container starts. Typical HPC codes are tuned to raw hardware so they assume full ownership of the hardware anyways. When HPC was developing 30 years ago we didnt have clean ways to avoid overheads in the regime of 10k nodes. Instead we got parallel filesystems, caching, and shell, with module, which technically did the job for reproducible runs at a huge human cost.

Re: Reflections on 30 years of HPC programming

#123

Post by suuuuuuuu »

ablob wrote: post #106
suuuuuuuu wrote: post #64

Earlier quoted context omitted.

Sorry, I wasn't aware of these developments (having abandoned CUDA for hardware-agnostic solutions before 2020). It doesn't change my point anyway, if it's specific to a single vendor. I'm extremely dubious that such an opaque abstraction can actually solve the (true) problem. "Not having to write CUDA" is not enough - how do you tune performance? Parallelization strategies, memory prefetching and arrangement in on-c…

So what would be such an HPC language that you're so fond of? A quick web search reveals only languages that use C++/CUDA code as a back end (python), are new and experimental (Julia) or FORTRAN. For what you're talking about none seem all to good, so you've peaked my curiosity.

See https://arxiv.org/abs/2512.17101. I've used some of the tools in the stack they describe (and see Sec 2 for an overview of others). JAX/XLA/etc. are somewhat similar, though still without user control over transformations.

Perhaps part of the reason for the bad takes in this thread is due to taking "language" overly literally (perhaps also the fault of the linked blog post itself). I think one thesis of the above tooling is that, when tuning and generating code (CUDA, OpenCL, what have you) at runtime, the best "languages" for these abstractions are, amusingly, scripting languages like Python. Having CUDA/etc. as a back end without having to hand-write/-transform/-optimize it is indeed the point.

Re: Reflections on 30 years of HPC programming

#124

Post by suuuuuuuu »

ablob wrote: post #108
hpcdude wrote: post #71

HPCdude here, and this is a mostly correct article, but here are what it misses: 1) It mentions in passing the hardware abstraction not being as universal as it seemed. This is more and more true, once we started doing fpgas, then asics, and as ARM and other platforms starting making headway, it fractured things a bit. GPUs too: I'm still a bit upset about CUDA winning over OpenCL, but Vulkan compute gives me hope. I…

afaik, you always have to break open abstractions for more performance. If you ignore cache-levels in your program you're gonna have a bad time - and depending on the system the layout (and with it how you should use it) is different. The same is true for how machines are interconnected. Depending on the wiring you have different throughput-values when sharing data between nodes. The whole area screams "not universal…

I think people working on these abstractions would claim that an appropriate abstraction does involve the exact features you mention. Cache hierarchies and vectorization are common to CPUs and GPUs - in some sense, the numbers parametrizing them are all that differs. With a good abstraction, this machine-tuning can be automated.

Re: Reflections on 30 years of HPC programming

#125

Post by m_mueller »

_flux wrote: post #70
m_mueller wrote: post #65

Earlier quoted context omitted.

btw. Fortran is implicitly behaving as "restrict" by default, which makes sense together with its intuitive "intent" system for function/subroutine arguments. This is one of the biggest reasons why it's still so popular in HPC - scientists can pretty much just write down their equations, follow a few simple rules (e.g. on storage order) and out comes fairly performant machine code. Doing the same (a 'naive' first imp…

Oh I actually had some editing mistake, I meant to say that also Rust has restrict by default, by virtue of all references being unique xor readonly. As I understand it, the Fortran compiler just expects your code to respect the "restrictness", it doesn't enforce it.

So that's where the intent system comes in (an argument can be in/out/inout) as well as the built-in array sizes, because it allows you to express what you want and then the compiler will enforce it. In Fortran you kinda have to work hard to invade the memory of one array from another, as they are allocated as distinct memory regions with their own space from the beginning. Pointer math is almost never necessary. Because there is built-in support for multidim arrays and array lengths, arrays are internally anyways built as flat memory regions, the same way you'd do it in C-arrays for good performance (i.e. cache locality), but with simple indices to address them. This then makes it unnecessary to treat memory as aliased by default.

Honestly, I still don't get why people have built up all these complex numerics frameworks in C and C++. Just use Fortran - it's built for exactly this usecase, and scientists will still be able to read your code without a CS degree. In fact, they'll probably be the ones writing it in the first place.

Re: Reflections on 30 years of HPC programming

#126

Post by pklausler »

m_mueller wrote: post #125
_flux wrote: post #70

Earlier quoted context omitted.

Oh I actually had some editing mistake, I meant to say that also Rust has restrict by default, by virtue of all references being unique xor readonly. As I understand it, the Fortran compiler just expects your code to respect the "restrictness", it doesn't enforce it.

So that's where the intent system comes in (an argument can be in/out/inout) as well as the built-in array sizes, because it allows you to express what you want and then the compiler will enforce it. In Fortran you kinda have to work hard to invade the memory of one array from another, as they are allocated as distinct memory regions with their own space from the beginning. Pointer math is almost never necessary. Bec…

There are good reasons to use Fortran, some having to do with the language and many to do with legacy codes. These have to be balanced with the good reasons to avoid using Fortran for new development, which also have to do with the language and its compilers.

Re: Reflections on 30 years of HPC programming

#127

Post by m_mueller »

pklausler wrote: post #126
m_mueller wrote: post #125

Earlier quoted context omitted.

So that's where the intent system comes in (an argument can be in/out/inout) as well as the built-in array sizes, because it allows you to express what you want and then the compiler will enforce it. In Fortran you kinda have to work hard to invade the memory of one array from another, as they are allocated as distinct memory regions with their own space from the beginning. Pointer math is almost never necessary. Bec…

There are good reasons to use Fortran, some having to do with the language and many to do with legacy codes. These have to be balanced with the good reasons to avoid using Fortran for new development, which also have to do with the language and its compilers.

To me it just boils down to using the right tool for each job. I definitely wouldn’t use Fortran for anything heavily using strings. One weakness is also the lack of meta programming support. But for numerical code to be run on a specific hardware, including GPU, it’s pretty close to perfect, especially also since NVIDIA invested into it.

Re: Reflections on 30 years of HPC programming

#128

Post by pklausler »

m_mueller wrote: post #127
pklausler wrote: post #126

Earlier quoted context omitted.

There are good reasons to use Fortran, some having to do with the language and many to do with legacy codes. These have to be balanced with the good reasons to avoid using Fortran for new development, which also have to do with the language and its compilers.

To me it just boils down to using the right tool for each job. I definitely wouldn’t use Fortran for anything heavily using strings. One weakness is also the lack of meta programming support. But for numerical code to be run on a specific hardware, including GPU, it’s pretty close to perfect, especially also since NVIDIA invested into it.

I’m glad you like it.

Re: Reflections on 30 years of HPC programming

#129

Post by bradcray »

guywithahat wrote: post #82

I like the idea of chapel, but I'm not sure I agree with a lot of their design choices. Some of the parallelization features seem like they just copied OpenMP without meaningfully improving on it. They also kept exceptions, which are generally on their way out, especially in compiled languages (Go, Rust, Zig, and while they exist in modern C++ they are introducing more ways to not use them). I think a new HPC languag…

I disagree with the characterization that Chapel's parallelization features copied OpenMP without improving upon it:

* Chapel's support for task parallelism predates OpenMP's (~2004 vs. ~2007, where Wikipedia cites Chapel's tasks as being inspiration for OpenMP's, along with Cilk and X10). Chapel's tasks are also arguably more general-purpose (akin to threads) in terms of their ability to synchronize, support data-driven producer/consumer patterns, etc.

* Chapel's forall loops are similar to OpenMP's loop-based parallelization pragmas, though OpenMP wasn't a source of inspiration in their design. Where OpenMP pragmas select from a menu of parallelization strategies baked into the specification and implementation, Chapel's forall loops invoke user-defined parallel iterators that permit abstracting a particular parallel pattern (say, multidimensional tiled iteration or tree traversal) into a named subroutine. These iterators can optionally be made methods of data structures and/or placed within libraries, and can be re-used across a program or multiple programs. One such library, DynamicIters, was community-contributed and specifically inspired by OpenMP's dynamic and guided scheduling strategies.

* Chapel supports parallel zippered iteration, in which two or more data structures and/or parallel iterators can be traversed in a coordinated manner.

* Chapel's parallelism can span multiple compute nodes via its shared namespace, which obviates the need for explicit communication; whereas OpenMP is limited to a single compute node or process unless mixed with MPI, SHMEM, or the like (and even then, OpenMP doesn't gain a cross-node view of parallel computation).

* In Chapel, parallelism can be expressed implicitly, for example, by passing an array argument to a subroutine or operator that is expecting a scalar (e.g., `var B = sin(A);` or `var C = A + B;`).

Post reply on HN