Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C++26 Shipped a SIMD Library Nobody Asked For

lucisqr.substack.com

Post reply on HN
Print view View original

91–100 of 170 posts

Re: C++26 Shipped a SIMD Library Nobody Asked For

#91

Post by raverbashing »

secondcoming wrote: post #88
raverbashing wrote: post #58

sigh C++ sits on that weird abstraction level where it wants to be a higher level language but it keeps grinding their gears on stuff like pointer sizes, pointer arithmetic or vector sizes and at the same time wants to keep being C compatible and needs that interface with the lower level world Now compare with how numpy does things: you care about the data size but not the implementation. Still, I didn't expect less…

numpy is a python wrapper over a C library written by people who have ground those gears

Yes but not all of them

It would be easy to push complexity up at the level of Numpy/Pytorch/Tensorflow but it mostly gets hidden

(also a lot of it relies on LAPACK which is Fortran - which kinda works with SIMD better than C/C++)

Re: C++26 Shipped a SIMD Library Nobody Asked For

#92

Post by camel-cdr »

meling wrote: post #67

Curious if people here have looked at the upcoming SIMD support in Go: https://go.dev/doc/go1.26#simd Currently experimental, but looks like the first Intel arch will arrive in the next release in about 3 months. They are also going to support a portable layer. Wondering what people here think about the approach the Go team is taking; I think they would appreciate more eyeballs on their design. (I’m not competent in…

Looks like that isn't a portable SIMD abstraction, but more similar to adding architecture-specific SIMD intrinsics support to go, with nicer syntax.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#93

Post by whizzter »

Agreed, fixed with vectors needs to be a language feature, better compile times and would solve issues for most people.

Personally, I think that like Clang way to adding GLSL like vectors and semantics would've gone a long way. SVE might be an elegant design, but in reality there are probably a multiple factor of game and other 3d code being written that needs vectors compared to other fields, and there limited vector sizes aren't really a problem.

And honestly, considering the story of AVX512.. with 512 bit vectors being removed from mainstream by Intel, do we really really need longer ones despite it being from a "scalable design"?

Re: C++26 Shipped a SIMD Library Nobody Asked For

#94

Post by mtklein »

pixelesque wrote: post #71
jandrewrogers wrote: post #3

I have written a lot of SIMD for both x86 and ARM over many years and many microarchitectures. Every abstraction, including autovectorization, is universally pretty poor outside of narrow cases because they don’t (and mostly can’t) capture what is possible with intrinsics and their rather extreme variation across microarchitectures. If I want good results, I have to write intrinsics. No library can optimally generate…

Yep, same here and agree. Compilers have definitely got better though: another issue in the past (maybe still is to a degree? although compilers have got a lot better at this in the past 15 years, but it used to be one of the things only Intel's ICC actually got right), that if you wrapped the base-level '__m128' or 'float32x4_t' in a struct/union in order to provide some abstraction, the compiler would often lose tr…

Part of that could be ABI constraints. There are some surprising calling convention differences between a vector and a struct or union with vectors in it, and they vary platform to platform. E.g. on ARM a struct with two 128-bit vectors will pass in two registers where on x86 it must pass via the stack.

Using __attribute__ to tweak calling conventions can often really clean this up, but that's just as obscure and non-portable as the problem it fixes. So you either end up writing weird non-portable code one way or weird non-portable code another... Code working with these types doesn't get to benefit from zero-cost abstraction to the degree we're used to with normal scalar code.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#95

Post by camel-cdr »

whizzter wrote: post #93

Agreed, fixed with vectors needs to be a language feature, better compile times and would solve issues for most people. Personally, I think that like Clang way to adding GLSL like vectors and semantics would've gone a long way. SVE might be an elegant design, but in reality there are probably a multiple factor of game and other 3d code being written that needs vectors compared to other fields, and there limited vecto…

In GPUs GLSL like types compile down to what basically is variable length SIMD. A vec4 doesn't get compiled to a SIMD vector with four floats, but rather to four SIMD vectors, each containing N FP32 elements (usually 32 or 64).

Look at what this simple shader compiles down to on RGA: https://godbolt.org/z/4GrfY61vf

Re: C++26 Shipped a SIMD Library Nobody Asked For

#96

Post by himata4113 »

Something everyone is missing is that this is just feature parity with other langauges, this is not c++ specific. They added it because other languages have these compiler hints which are then (usually) used in llvm as opaque types for 'smarter' optimizations. Hand rolled code will still be better, but there are very niche instances where you want llvm to know this information, especially important when you don't care about performance, but you care about data integrity and obfuscation.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#97

Post by hansvm »

exceptione wrote: post #89
jandrewrogers wrote: post #3

I have written a lot of SIMD for both x86 and ARM over many years and many microarchitectures. Every abstraction, including autovectorization, is universally pretty poor outside of narrow cases because they don’t (and mostly can’t) capture what is possible with intrinsics and their rather extreme variation across microarchitectures. If I want good results, I have to write intrinsics. No library can optimally generate…

Do you say that from the perspective of compiled languages? I hear good things about .net core wrt SIMD, but that has the advantage it can decide at JIT.

I'm not the person you're asking, but I share that opinion for both compiled languages and JIT solutions, including .net core specifically. All but the most trivial use cases can't be autovectorized, by JIT or otherwise. One of the recent things I worked on (reed-solomon decoding) offers basically zero opportunities for autovectorization unless the compiler reinterprets certain scalar loops as dedicated galois instructions on AVX512F hardware, but that optimization isn't implemented, it wouldn't help other architectures anyway, and it's still 10x slower than a well thought out vectorized approach.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#98

Post by janwas »

fweimer wrote: post #85
janwas wrote: post #75

Earlier quoted context omitted.

Have you considered our Highway library? Runtime dispatch need not be a PITA :) It's basically portable intrinsics, and a much more complete set (>300) than the ~50 in std.

Does it have fallback paths for everything, though? Scalar if necessary? Projects that depend on Highway drop support for CPUs not listed in the Highway documentation, saying that they can't support these CPUs because they are incompatible with Highway: https://google.github.io/highway/en/master/README.html#curre... Are these projects somehow mistaken?

Yes, the EMU128 target is scalar only, with for loops. This is a fun way to see how well autovectorization works, with the same source code. That works on any CPU. Curious which projects have such concerns, any link?

Re: C++26 Shipped a SIMD Library Nobody Asked For

#99

Post by SkiFire13 »

camel-cdr wrote: post #90
SkiFire13 wrote: post #76

Earlier quoted context omitted.

> The problem is that you're better off by defining SIMD friendly data structures and letting the compiler figure it out than by hand coding the actual SIMD operations. This will work only for the most basic SIMD usages. > CPU vendors must offer a basic set of vector instructions that is supported on all architectures. This will take decades because you cannot change existing architectures/processors.

> This will take decades because you cannot change existing architectures/processors. I think once, AVX-512, SVE and RVV are wide spread enough, you'll have a rather powerfull baselevel you can target. But this will take a lot of time.

> AVX-512

Which subset though? Some of them are not supported by some recent CPUs (e.g. 2024).

Not to mention Alder Lake not supporting AVX512.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#100

Post by hansvm »

mattip wrote: post #16
jandrewrogers wrote: post #8

Earlier quoted context omitted.

For some algorithms you have to compromise the data layout for compatibility across the widest number of microarchitectures by nerfing the performance on advanced SIMD microarchitectures working on the same data structures. There really isn’t a way to square that circle. You can make it portable or you can make it optimal, and the performance gap across those two implementations can be vast. In the 15-20 years I’ve b…

NumPy has a whole dispatch mechanism to deal with the tradeoffs. The main problem is code bloat: how many microarchitectures are you going to support with dispatch at runtime?

Numpy is interesting in that regard since its dispatch mechanism adds up to a lot of overhead. There are a lot of problems where a naive list comprehension is faster, even when SIMD could be used to great effect.
Post reply on HN