Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C++26 Shipped a SIMD Library Nobody Asked For

lucisqr.substack.com

Post reply on HN
Print view View original

11–20 of 170 posts

Re: C++26 Shipped a SIMD Library Nobody Asked For

#11

Post by cortesoft »

jandrewrogers wrote: post #3

I have written a lot of SIMD for both x86 and ARM over many years and many microarchitectures. Every abstraction, including autovectorization, is universally pretty poor outside of narrow cases because they don’t (and mostly can’t) capture what is possible with intrinsics and their rather extreme variation across microarchitectures. If I want good results, I have to write intrinsics. No library can optimally generate…

Is this a technical impossibility or just it hasn't been done yet? Could a library support generating intrinsics for a large set of architectures?

Re: C++26 Shipped a SIMD Library Nobody Asked For

#12

Post by AlotOfReading »

mgaunard wrote: post #7

I made the first proposal to the C++ standard committee to introduce SIMD in 2011, before Matthias Kretz got involved with his own version (which is what became std::simd). This was based on what eventually became Eve (mentioned in the article). Back then, it was rejected, for the same arguments that people are making today, such as not mapping to SVE well, having a separate way to express control flow etc. There was…

Trying to abstract over SVE with a SIMD library is a bit of a fool's errand. The intended programming model is just too different from traditional ISAs, and there are algorithms that are nearly impossible to write efficiently for it. All the ones I've seen wrap it up as a bastardized fixed length ISA, and even ARM's own guidance basically recommends that approach.

Frankly, the length agnostic stuff is a mistake that I hope hardware designers will eventually see the light on, like delay slots.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#13

Post by loeg »

cortesoft wrote: post #11
jandrewrogers wrote: post #3

I have written a lot of SIMD for both x86 and ARM over many years and many microarchitectures. Every abstraction, including autovectorization, is universally pretty poor outside of narrow cases because they don’t (and mostly can’t) capture what is possible with intrinsics and their rather extreme variation across microarchitectures. If I want good results, I have to write intrinsics. No library can optimally generate…

Is this a technical impossibility or just it hasn't been done yet? Could a library support generating intrinsics for a large set of architectures?

Google Highway gets mentioned in the article.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#14

Post by mattip »

cortesoft wrote: post #11
jandrewrogers wrote: post #3

I have written a lot of SIMD for both x86 and ARM over many years and many microarchitectures. Every abstraction, including autovectorization, is universally pretty poor outside of narrow cases because they don’t (and mostly can’t) capture what is possible with intrinsics and their rather extreme variation across microarchitectures. If I want good results, I have to write intrinsics. No library can optimally generate…

Is this a technical impossibility or just it hasn't been done yet? Could a library support generating intrinsics for a large set of architectures?

There is google’s highway, that provides an abstraction layer. It is used by NumPy.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#15

Post by rbanffy »

mgaunard wrote: post #7

I made the first proposal to the C++ standard committee to introduce SIMD in 2011, before Matthias Kretz got involved with his own version (which is what became std::simd). This was based on what eventually became Eve (mentioned in the article). Back then, it was rejected, for the same arguments that people are making today, such as not mapping to SVE well, having a separate way to express control flow etc. There was…

To me it’s clear adding the ability to express intent to parallelise is the Right Thing. This is the only way the compiler can actually know what you want it to do.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#16

Post by mattip »

jandrewrogers wrote: post #8
mgaunard wrote: post #6

Earlier quoted context omitted.

For me the main issue is that if you're serious about SIMD, you need to use a state-of-the-art library and can't rely on some standard library whose quality is variable, unreliable, and which is by design always behind.

For some algorithms you have to compromise the data layout for compatibility across the widest number of microarchitectures by nerfing the performance on advanced SIMD microarchitectures working on the same data structures. There really isn’t a way to square that circle. You can make it portable or you can make it optimal, and the performance gap across those two implementations can be vast. In the 15-20 years I’ve b…

NumPy has a whole dispatch mechanism to deal with the tradeoffs. The main problem is code bloat: how many microarchitectures are you going to support with dispatch at runtime?

Re: C++26 Shipped a SIMD Library Nobody Asked For

#17

Post by jandrewrogers »

mpyne wrote: post #9
jandrewrogers wrote: post #3

I have written a lot of SIMD for both x86 and ARM over many years and many microarchitectures. Every abstraction, including autovectorization, is universally pretty poor outside of narrow cases because they don’t (and mostly can’t) capture what is possible with intrinsics and their rather extreme variation across microarchitectures. If I want good results, I have to write intrinsics. No library can optimally generate…

> I think a legitimate criticism is that it is unclear who std::simd is for. I think it's for people like me, who recognize that depending on the dataset that a lot of performance is left on the table for some datasets when you don't take advantage of SIMD, but are not interested in becoming experts on intrinsics for a multitude of processor combinations. Having a way to be able to say "flag bytes in this buffer matc…

You raise some good points. I think a lot about how to make SIMD more accessible, and spend an inordinate amount of time experimenting with abstractions, because I’ve experienced its many inadequacies.

The design of the intrinsics libraries do themselves no favors and there are many inconsistencies. Basic things could be made more accessible but are somewhat limited by a requirement for C compatibility. This is something a C++ standard can actually address — it can be C++ native, which can hide many things. Hell, I have my own libraries that clean this up by thinly wrapping the existing intrinsics, improving their conciseness and expressiveness for common use cases. It significantly improves the ergonomics.

An argument I would make though is that the lowest common denominator cases that are actually portable are almost exactly the cases that auto-vectorization should be able to address. Auto-vectorization may not be good enough to consistently address all of those cases today but you can see a future where std::simd is essentially vestigial because auto-vectorization subsumes what it can do but it can’t be leveled up to express more than what auto-vectorization can see due to limitations imposed by portability requirements.

The other argument is that SIMD is the wrong level of abstraction for a library. Depending on the microarchitecture, the optimal code using SIMD may be an entirely different data structure and algorithm, so you are swapping out SIMD details at a very abstract macro level, not at the level of abstraction that intrinsics and auto-vectorization provide. You miss a lot of optimization if you don’t work a couple levels up.

SIMD abstraction and optimization is deeply challenging within programming languages designed around scalar ALU operators. We can’t even fully abstract the expressiveness of modern scalar ALUs across microarchitectures because programming languages don’t define a concept that maps to the capabilities of some modern ALUs.

That said, I love that silicon has become so much more expressive.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#18

Post by [deleted] »

jandrewrogers wrote: post #8
mgaunard wrote: post #6

Earlier quoted context omitted.

For me the main issue is that if you're serious about SIMD, you need to use a state-of-the-art library and can't rely on some standard library whose quality is variable, unreliable, and which is by design always behind.

For some algorithms you have to compromise the data layout for compatibility across the widest number of microarchitectures by nerfing the performance on advanced SIMD microarchitectures working on the same data structures. There really isn’t a way to square that circle. You can make it portable or you can make it optimal, and the performance gap across those two implementations can be vast. In the 15-20 years I’ve b…

[deleted]

Re: C++26 Shipped a SIMD Library Nobody Asked For

#19

Post by jandrewrogers »

paulddraper wrote: post #10
jandrewrogers wrote: post #3

I have written a lot of SIMD for both x86 and ARM over many years and many microarchitectures. Every abstraction, including autovectorization, is universally pretty poor outside of narrow cases because they don’t (and mostly can’t) capture what is possible with intrinsics and their rather extreme variation across microarchitectures. If I want good results, I have to write intrinsics. No library can optimally generate…

> I think a legitimate criticism is that it is unclear who std::simd is for It's for people that don't use SIMD today. SIMD is hard, or at least nuanced and platform-dependant. To say that std::simd doesn't lower the learning curve is intellectually dishonest. --- Despite the title, the primary criticism of the article is that the compilers' auto-vectorizers have improved better than the current shipped stdlib versio…

My criticism could mostly be summarized similarly. The scope of what a portable std::simd can do is almost exactly the scope that you would expect auto-vectorization to subsume over time. SIMD, to the extent it is covered by std::simd, is the part of SIMD that should be pretty simple to learn.

There isn’t an obvious path to elevate it above what auto-vectorization should theoretically be capable of in a portable way. This leads to a potential long-term outcome where std::simd is essentially a no-op because scalar code is automagically converted into the equivalent and it is incapable of supporting more sophisticated SIMD code.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#20

Post by mgaunard »

jandrewrogers wrote: post #8
mgaunard wrote: post #6

Earlier quoted context omitted.

For me the main issue is that if you're serious about SIMD, you need to use a state-of-the-art library and can't rely on some standard library whose quality is variable, unreliable, and which is by design always behind.

For some algorithms you have to compromise the data layout for compatibility across the widest number of microarchitectures by nerfing the performance on advanced SIMD microarchitectures working on the same data structures. There really isn’t a way to square that circle. You can make it portable or you can make it optimal, and the performance gap across those two implementations can be vast. In the 15-20 years I’ve b…

For Boost.SIMD (which is what became Eve), a large part of what we did to tackle those problems was building an overload dispatching system so that we could easily inject increasingly specialized implementations depending on the types and instruction set available, in such a way that operations could combine efficiently.

That, however, performed quite poorly at compile-time, and was not really ODR-safe (forceinline was used as a workaround). At least one of the forks moved to using a dedicated meta-language and a custom compiler to generate the code instead. There are better ways to do that in modern C++ now.

We also focused on higher-level constructs trying to capture the intent rather than trying to abstract away too low-level features; some of the features were explicitly provided as kernels or algorithms instead of plain vector operations.

Post reply on HN