Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C++26 Shipped a SIMD Library Nobody Asked For

lucisqr.substack.com

Post reply on HN
Print view View original

21–30 of 170 posts

Re: C++26 Shipped a SIMD Library Nobody Asked For

#21

Post by duped »

mpyne wrote: post #9
jandrewrogers wrote: post #3

I have written a lot of SIMD for both x86 and ARM over many years and many microarchitectures. Every abstraction, including autovectorization, is universally pretty poor outside of narrow cases because they don’t (and mostly can’t) capture what is possible with intrinsics and their rather extreme variation across microarchitectures. If I want good results, I have to write intrinsics. No library can optimally generate…

> I think a legitimate criticism is that it is unclear who std::simd is for. I think it's for people like me, who recognize that depending on the dataset that a lot of performance is left on the table for some datasets when you don't take advantage of SIMD, but are not interested in becoming experts on intrinsics for a multitude of processor combinations. Having a way to be able to say "flag bytes in this buffer matc…

> it's a PITA to differentiate the utility of hundreds of different vpaddcfoolol instructions

This is one complaint I toss back at Intel and AMD.

If an instruction/intrinsic is universally worse than the P90/P95/P99 use case where it's going to be used to another set of instrinsics, then it shouldn't exist. Stop wasting the die space and instruction decode on it, if not only the developer time wasted finding out that your dot product instruction is useless.

There are a lot of smart people that have worked on compilers, optimized subroutines for LAPACK/BLAS, and designed the decoders and hardware. A lot of that effort is wasted because no one knows how to program these weird little machines. A little manual on "here's how to program SIMD, starting from linear algebra basics" would be worth more to Intel than all the money they've wasted trying to improve autovectorization passes in ICC and now, LLVM.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#22

Post by synergy20 »

jandrewrogers wrote: post #3

I have written a lot of SIMD for both x86 and ARM over many years and many microarchitectures. Every abstraction, including autovectorization, is universally pretty poor outside of narrow cases because they don’t (and mostly can’t) capture what is possible with intrinsics and their rather extreme variation across microarchitectures. If I want good results, I have to write intrinsics. No library can optimally generate…

what about Google highway project?

Re: C++26 Shipped a SIMD Library Nobody Asked For

#23

Post by jandrewrogers »

cortesoft wrote: post #11
jandrewrogers wrote: post #3

I have written a lot of SIMD for both x86 and ARM over many years and many microarchitectures. Every abstraction, including autovectorization, is universally pretty poor outside of narrow cases because they don’t (and mostly can’t) capture what is possible with intrinsics and their rather extreme variation across microarchitectures. If I want good results, I have to write intrinsics. No library can optimally generate…

Is this a technical impossibility or just it hasn't been done yet? Could a library support generating intrinsics for a large set of architectures?

The full scope of what SIMD is used for is much larger than parallelizing evaluation of numeric types and algorithms.

For example, it is used for parallel evaluation of complex constraints on unrelated types simultaneously while packed into a single vector. Think a WHERE clause on an arbitrary SQL schema evaluated in full parallel in a handful of clock cycles. SIMD turns out to be brilliant for this but it looks nothing like auto-vectorization.

None of the SIMD libraries like Google Highway cover this case.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#24

Post by kent-tokyo »

jandrewrogers wrote: post #3

I have written a lot of SIMD for both x86 and ARM over many years and many microarchitectures. Every abstraction, including autovectorization, is universally pretty poor outside of narrow cases because they don’t (and mostly can’t) capture what is possible with intrinsics and their rather extreme variation across microarchitectures. If I want good results, I have to write intrinsics. No library can optimally generate…

[dead]

Re: C++26 Shipped a SIMD Library Nobody Asked For

#25

Post by countWSS »

GCC already solved it: https://gcc.gnu.org/onlinedocs/gcc/Vector-Extensions.html The operations behave like C++ valarrays. Addition is defined as the addition of the corresponding elements of the operands. For example, in the code below, each of the 4 elements in a is added to the corresponding 4 elements in b and the resulting vector is stored in c.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#26

Post by camel-cdr »

jandrewrogers wrote: post #8
mgaunard wrote: post #6

Earlier quoted context omitted.

For me the main issue is that if you're serious about SIMD, you need to use a state-of-the-art library and can't rely on some standard library whose quality is variable, unreliable, and which is by design always behind.

For some algorithms you have to compromise the data layout for compatibility across the widest number of microarchitectures by nerfing the performance on advanced SIMD microarchitectures working on the same data structures. There really isn’t a way to square that circle. You can make it portable or you can make it optimal, and the performance gap across those two implementations can be vast. In the 15-20 years I’ve b…

The data layout can often be done dynamically based on your target architecture.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#27

Post by camel-cdr »

jandrewrogers wrote: post #23
cortesoft wrote: post #11

Earlier quoted context omitted.

Is this a technical impossibility or just it hasn't been done yet? Could a library support generating intrinsics for a large set of architectures?

The full scope of what SIMD is used for is much larger than parallelizing evaluation of numeric types and algorithms. For example, it is used for parallel evaluation of complex constraints on unrelated types simultaneously while packed into a single vector. Think a WHERE clause on an arbitrary SQL schema evaluated in full parallel in a handful of clock cycles. SIMD turns out to be brilliant for this but it looks noth…

I don't quite get how something like highway doesn't cover this, while intrinsics do.

Can you explain the usecase more concretely?

Re: C++26 Shipped a SIMD Library Nobody Asked For

#28

Post by groundzeros2015 »

countWSS wrote: post #25

GCC already solved it: https://gcc.gnu.org/onlinedocs/gcc/Vector-Extensions.html The operations behave like C++ valarrays. Addition is defined as the addition of the corresponding elements of the operands. For example, in the code below, each of the 4 elements in a is added to the corresponding 4 elements in b and the resulting vector is stored in c.

Thanks!

Re: C++26 Shipped a SIMD Library Nobody Asked For

#29

Post by camel-cdr »

AlotOfReading wrote: post #12
mgaunard wrote: post #7

I made the first proposal to the C++ standard committee to introduce SIMD in 2011, before Matthias Kretz got involved with his own version (which is what became std::simd). This was based on what eventually became Eve (mentioned in the article). Back then, it was rejected, for the same arguments that people are making today, such as not mapping to SVE well, having a separate way to express control flow etc. There was…

Trying to abstract over SVE with a SIMD library is a bit of a fool's errand. The intended programming model is just too different from traditional ISAs, and there are algorithms that are nearly impossible to write efficiently for it. All the ones I've seen wrap it up as a bastardized fixed length ISA, and even ARM's own guidance basically recommends that approach. Frankly, the length agnostic stuff is a mistake that…

> Trying to abstract over SVE with a SIMD library is a bit of a fool's errand

It reallt isn't. You just make the default SIMD-width agnostic and anything less portable opt-in.

You can still specialize for a specific width pn scalabe vector ISAs.

> The intended programming model is just too different from traditional ISAs, and there are algorithms that are nearly impossible to write efficiently for it.

Such as?

> All the ones I've seen wrap it up as a bastardized fixed length ISA, and even ARM's own guidance basically recommends that approach.

google highway doesn't. And while Arm is stuck with 128-bit SVE, because they alsp have to implement NEON as fast as possible to be competitive, RVV already has a large diversitly of hardware with different vector length available 128,256,512,1024.

Post reply on HN