Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Reflections on 30 years of HPC programming

chapel-lang.org

Post reply on HN
Print view View original

51–60 of 129 posts

Re: Reflections on 30 years of HPC programming

#51

Post by j4k0bfr »

Joel_Mckay wrote: post #49
j4k0bfr wrote: post #48

Earlier quoted context omitted.

I'm talking almost exactly about this haha. Flight software representation!! Although I have no experience programming FPGAs, I hope to gain some soon. They seem like the ultimate solution to my IO woes.

A bit legacy these days, but I liked the old Zynq 7020 dual core ARM with reasonable LUT counts. https://www.youtube.com/watch?v=FujoiUMhRdQ https://github.com/Spyros-2501/Z-turn-Board-V2-Diary https://www.youtube.com/@TheDevelopmentChannel/playlists https://myirtech.com/list.asp?id=708 The Debian Linux example includes how to interface hardware ports. Almost always better to go with a simpler mcu if one can get away…

No way, was looking at the Z-7045 for a work project literally today. And yep I agree, simpler solutions have simpler problems lol. Thanks for the recommendation, I'll give it a look!

Re: Reflections on 30 years of HPC programming

#52

Post by suuuuuuuu »

fcanesin wrote: post #31
suuuuuuuu wrote: post #28

Earlier quoted context omitted.

If you think C++ is the best here, then I don't think you've actually worked in this space nor appreciated the actual problems these languages try to solve. In particular because you can't program accelerators with C++. Memory bandwidth is often the problem, yes. Language abstractions for performance aim to, e.g., automatically manage caches (that must be handled manually in performant GPU code, for instance) with op…

Wait what!? I have been programming CUDA since 2009 and specifically remember it being pushed to C++ as main development language for the first few years, after a brief "CUDA C extension" period.

CUDA is not C++. CUDA for GPU kernels is its own language. That's the actual problem requiring new languages or abstractions.

Re: Reflections on 30 years of HPC programming

#53

Post by ameliaquining »

_flux wrote: post #29
lugu wrote: post #24

Earlier quoted context omitted.

Parent talks about new languages, as per the article Fortran or C doing fine. I speculate the benefit of C++ over Rust how it let programmers instruct the compiler of warranty that goes beyong the initial semantic of the language. See __restrict, __builtin_prefetch and __builtin_assume_aligned. The programming language is a space for conversations between compiler builders and hardware designers.

I believe __restrict, and __builtin_prefetch/__builtin_assume are compiler extensions, not part of the C++ language as is, and different compilers implement (or don't) these differently. The rust compiler actually has similar things, but they're not available in stable builds. I suppose there are some issues if principle why not to include them in stable. E.g: https://doc.rust-lang.org/std/intrinsics/fn.prefetch_read…

Rust already has __restrict; it is spelled &mut and is one of the most fundamental parts of the language. The key difference, of course, is that it's checked by the compiler, so is useful for correctness and not just performance. Also, for a long time it wasn't used for optimization, because the corresponding LLVM feature (noalias) was full of miscompilation bugs, because not that much attention was being paid to it, because hardly anyone actually uses restrict in C or __restrict in C++. But those days are finally over.

__builtin_assume is available on stable (though of course it's unsafe): https://doc.rust-lang.org/std/hint/fn.assert_unchecked.html

There's an open issue to stabilize the prefetch APIs: https://github.com/rust-lang/rust/issues/146941 As is usually the case when a minor standard-library feature remains unstable, the primary reason is that nobody has found the problem urgent enough to put in the required work to stabilize it. (There's an argument that this process is currently too inefficient, but that's a separate issue.) In the meantime, there are third-party libraries available that use inline assembly to offer this functionality, though this means they only support a couple of the most popular architectures.

Re: Reflections on 30 years of HPC programming

#54

Post by suuuuuuuu »

pjmlp wrote: post #35
suuuuuuuu wrote: post #28

Earlier quoted context omitted.

If you think C++ is the best here, then I don't think you've actually worked in this space nor appreciated the actual problems these languages try to solve. In particular because you can't program accelerators with C++. Memory bandwidth is often the problem, yes. Language abstractions for performance aim to, e.g., automatically manage caches (that must be handled manually in performant GPU code, for instance) with op…

What?!? NVidia designs CUDA hardware specifically for the C++ memory model, they went through the trouble to refactor their original hardware across several years, so that all new cards would follow this model, even if PTX was designed as polyglot target. Additionally, ISO C++ papers like senders/receivers are driven by NVidia employees working on CUDA.

CUDA is not C++. CUDA for GPU kernels is its own language. That's the actual problem requiring new languages or abstractions.

Re: Reflections on 30 years of HPC programming

#55

Post by ozgrakkurt »

lugu wrote: post #24
bruce343434 wrote: post #7

Earlier quoted context omitted.

What does it mean to be friendly to memory bandwidth, and why does C++ excel at it, over, say, Fortran or C or Rust?

Parent talks about new languages, as per the article Fortran or C doing fine. I speculate the benefit of C++ over Rust how it let programmers instruct the compiler of warranty that goes beyong the initial semantic of the language. See __restrict, __builtin_prefetch and __builtin_assume_aligned. The programming language is a space for conversations between compiler builders and hardware designers.

It is just super unpleasant to write low level software in rust.

There is a colossal ergonomics difference if you compare using clang vs rust to writing a hashmap for example.

C compilers just have everything you can think of because everythin is first implemented there.

Using anything else just seems kind of pointless. I understand new languages do have benefits but I don't believe language matters that much really.

The person who writes that garbage pointer soup in C write Arc + multi threaded + macro garbage soup in Rust.

Re: Reflections on 30 years of HPC programming

#56

Post by pjmlp »

RhysU wrote: post #47
pjmlp wrote: post #36

Earlier quoted context omitted.

The authors are aware, as the Chapel compiler makes use of LLVM.

The author's framing "we have failed" suggests otherwise. This section, https://chapel-lang.org/blog/posts/30years/#ok-then-why , does not mention libraries at all.

Not really, you should actually read that section a few times as well.

> A fact of life in HPC is that the community has many large, long-lived codes written in languages like Fortran, C, and C++ that remain important. Such codes keep those languages at the forefront of peoples’ minds and sometimes lead to the belief that we can’t adopt new languages.

> In large part because of the previous point, our programming notations tend to take a bottom-up approach. “What does this new hardware do, and how can we expose it to the programmer from C/C++?” The result is the mash-up of notations that we have today, like C++, MPI, OpenMP, and CUDA. While they allow us to program our systems, and are sufficient for doing so, they also leave a lot to be desired as compared to providing higher-level approaches that abstract away the specifics of the target hardware.

Nothing there suggests the languages don't improve, especially anyone that follows ISO knows where many of improvements to Fortran, C and C++ are coming from.

For example, C++26 is probably going to get BLAS into the standard library, senders/receivers is being sponsored by CUDA money.

Another thing you missed from the author background, is that Chapel is sponsored by HPE and Intel, and one of the main targets are HPE Cray EX/XC systems, they know pretty well what is happening.

Re: Reflections on 30 years of HPC programming

#57

Post by pjmlp »

suuuuuuuu wrote: post #54
pjmlp wrote: post #35

Earlier quoted context omitted.

What?!? NVidia designs CUDA hardware specifically for the C++ memory model, they went through the trouble to refactor their original hardware across several years, so that all new cards would follow this model, even if PTX was designed as polyglot target. Additionally, ISO C++ papers like senders/receivers are driven by NVidia employees working on CUDA.

CUDA is not C++. CUDA for GPU kernels is its own language. That's the actual problem requiring new languages or abstractions.

Says those that don't know CUDA.

You can program CUDA in standard C++20, with CUDA libraries hidding the language extensions.

I love when C and C++ dialects are C and C++ when it matters, and not when it doesn't help to sell the ideas being portrayed.

Re: Reflections on 30 years of HPC programming

#59

Post by Joel_Mckay »

jacquesm wrote: post #50
Joel_Mckay wrote: post #20

Earlier quoted context omitted.

People did try to create an OTP in HDL at one point. And Erlang has already run many telecom infrastructures for decades. Surprising given how fragile the multi-host implementation has proven. Erlang/Elixir are neat languages, and right next to Julia for fun. =3

Yes, it's absolutely amazing. But I'm intimately familiar with how the Erlang VM works and it would seem to me to be a very bad match for HPC on the number crunching side, though it likely would do quite well on the orchestration side but that would require the people writing the rest of the code to change their way of working completely. And given how much of that is still F77 I highly doubt they would be willing to…

For simple AMQP, it has performed rather well for our use-cases over the years.

Haven't personally deployed this version yet. ymmv =3

https://www.rabbitmq.com/docs/quorum-queues

Re: Reflections on 30 years of HPC programming

#60

Post by kmaitreys »

Joeboy wrote: post #42
kmaitreys wrote: post #39

Earlier quoted context omitted.

Can you elaborate on this? Slightly concerned because I have written (and planning to write more) Rust HPC code

Maybe not what they meant, but Rust sometimes makes it tempting to just copy things rather than fighting the borrow checker. Whereas in C++ you're free to just pass pointers around and not worry about it until / unless your code crashes or gets exploited. Speaking authoritatively from my position as an incompetent C++ / Rust dev.

I see. Fortunately, I'm aware of that and I don't use clone (unless I intend to) as much. Borrow checker is usually not a problem when writing scientific/HPC code.

Because passing pointers isn't as ergonomic in Rust, I do things in arena-based way (for example setting up quadtrees or octrees). Is that part of the issue when it comes to memory bandwidth?

Post reply on HN