Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C++26 Shipped a SIMD Library Nobody Asked For

lucisqr.substack.com

Post reply on HN
Print view View original

71–80 of 170 posts

Re: C++26 Shipped a SIMD Library Nobody Asked For

#71

Post by pixelesque »

jandrewrogers wrote: post #3

I have written a lot of SIMD for both x86 and ARM over many years and many microarchitectures. Every abstraction, including autovectorization, is universally pretty poor outside of narrow cases because they don’t (and mostly can’t) capture what is possible with intrinsics and their rather extreme variation across microarchitectures. If I want good results, I have to write intrinsics. No library can optimally generate…

Yep, same here and agree.

Compilers have definitely got better though: another issue in the past (maybe still is to a degree? although compilers have got a lot better at this in the past 15 years, but it used to be one of the things only Intel's ICC actually got right), that if you wrapped the base-level '__m128' or 'float32x4_t' in a struct/union in order to provide some abstraction, the compiler would often lose track of this when passing the struct/union through functions (either by value or const ref), and would often end up 'spilling' (not entirely the correct terminology in this context, but...) the variable from registers, and just producing asm which ended up uselessly loading the variable again from a stack address further up the call stack, when it didn't actually need to do that. So that was the situation even when using intrinsics within custom wrappers.

From 2011 to around 2013 ICC seemed to be the only compiler on amd64 which wouldn't do this. If you passed the actual '__m128' down the function call chain instead, clang and gcc would then do the right thing.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#72

Post by imtringued »

AlotOfReading wrote: post #12
mgaunard wrote: post #7

I made the first proposal to the C++ standard committee to introduce SIMD in 2011, before Matthias Kretz got involved with his own version (which is what became std::simd). This was based on what eventually became Eve (mentioned in the article). Back then, it was rejected, for the same arguments that people are making today, such as not mapping to SVE well, having a separate way to express control flow etc. There was…

Trying to abstract over SVE with a SIMD library is a bit of a fool's errand. The intended programming model is just too different from traditional ISAs, and there are algorithms that are nearly impossible to write efficiently for it. All the ones I've seen wrap it up as a bastardized fixed length ISA, and even ARM's own guidance basically recommends that approach. Frankly, the length agnostic stuff is a mistake that…

I don't know how SVE works but I thought the point of it was to let implementations pick a larger size than the CPU supports and then get an automatic speedup from better processors with more vector lanes.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#73

Post by imtringued »

stephbook wrote: post #34
AlotOfReading wrote: post #12

Earlier quoted context omitted.

Trying to abstract over SVE with a SIMD library is a bit of a fool's errand. The intended programming model is just too different from traditional ISAs, and there are algorithms that are nearly impossible to write efficiently for it. All the ones I've seen wrap it up as a bastardized fixed length ISA, and even ARM's own guidance basically recommends that approach. Frankly, the length agnostic stuff is a mistake that…

I'm no C++ dev, but as an outsider, it sure reads like the whole "int is variable length" mistake again.

That's a mistake for ABI visible types, yes.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#74

Post by Pannoniae »

fithisux wrote: post #47

Why not just writing inline assembly is not enough? You optimize for a specific target. The problem is that you cannot be cross-platform. Sure. But that is why software is incremental. I write for my HW, not yours. You can write for yours. Make folders with implemntations x86_v1 x86_v2 arm64 riscv64 ... ... ... and include

sadly inline assembly is still at the ergonomics of "one compiler doesn't support it in x64 mode" and "you can choose between the readable syntax (which is a black box to the compiler) and the unreadable syntax (which can specify I/O/clobber regs)"

Re: C++26 Shipped a SIMD Library Nobody Asked For

#75

Post by janwas »

mpyne wrote: post #9
jandrewrogers wrote: post #3

I have written a lot of SIMD for both x86 and ARM over many years and many microarchitectures. Every abstraction, including autovectorization, is universally pretty poor outside of narrow cases because they don’t (and mostly can’t) capture what is possible with intrinsics and their rather extreme variation across microarchitectures. If I want good results, I have to write intrinsics. No library can optimally generate…

> I think a legitimate criticism is that it is unclear who std::simd is for. I think it's for people like me, who recognize that depending on the dataset that a lot of performance is left on the table for some datasets when you don't take advantage of SIMD, but are not interested in becoming experts on intrinsics for a multitude of processor combinations. Having a way to be able to say "flag bytes in this buffer matc…

Have you considered our Highway library? Runtime dispatch need not be a PITA :) It's basically portable intrinsics, and a much more complete set (>300) than the ~50 in std.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#76

Post by SkiFire13 »

imtringued wrote: post #68
portly wrote: post #54

Earlier quoted context omitted.

Don't let the best be the enemy of the good. I got amazing performance for swapping for-loops with some simple SIMD patterns. Moreover. By doing this. I noticed that the codebase started to become better shaped for performance as well. By writing SIMD patterns, you get into the mindset of tight, hot loops.

The problem is that you're better off by defining SIMD friendly data structures and letting the compiler figure it out than by hand coding the actual SIMD operations. If you wanted to explicitly opt into bundling/batching of operations, you wouldn't actually want to define a fixed register size. You'd want a data type that represents an arbitrarily sized register and exposes some across batch operations. Then the com…

> The problem is that you're better off by defining SIMD friendly data structures and letting the compiler figure it out than by hand coding the actual SIMD operations.

This will work only for the most basic SIMD usages.

> CPU vendors must offer a basic set of vector instructions that is supported on all architectures.

This will take decades because you cannot change existing architectures/processors.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#77

Post by janwas »

duped wrote: post #21
mpyne wrote: post #9

Earlier quoted context omitted.

> I think a legitimate criticism is that it is unclear who std::simd is for. I think it's for people like me, who recognize that depending on the dataset that a lot of performance is left on the table for some datasets when you don't take advantage of SIMD, but are not interested in becoming experts on intrinsics for a multitude of processor combinations. Having a way to be able to say "flag bytes in this buffer matc…

> it's a PITA to differentiate the utility of hundreds of different vpaddcfoolol instructions This is one complaint I toss back at Intel and AMD. If an instruction/intrinsic is universally worse than the P90/P95/P99 use case where it's going to be used to another set of instrinsics, then it shouldn't exist. Stop wasting the die space and instruction decode on it, if not only the developer time wasted finding out that…

:) I agree a tutorial would be helpful. We are working on one with Fastcode.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#78

Post by tialaramex »

pwdisswordfishq wrote: post #56
jcranmer wrote: post #38

If you thought std::simd was a library nobody asked for, just wait until you hear about . I feel like half the people looking forward to that think they're just going to get standard C++ bindings to LAPACK, when instead they're probably going to get an unoptimized, slapdash implementation of LAPACK written by people who aren't good at BLAS. As for SIMD itself, designing a good SIMD library is difficult because there…

Just wait until you hear about std::hive. The work of one obsessive author, who never gave a good explanation for why the thing needed to be in the standard library instead of an external one. The committee was apathetic about the proposal and kept bringing up various trivial issues, in a clear attempt to stall him, but he refused to take the hint. So eventually they relented. Outside coverage I have seen so far seem…

I feel like std::hive fits right in to the C++ stdlib group of collections

The least stupid is std::vector which is just the typical O(1) amortized growable array type found in most modern languages, with a mediocre API. 8/10 could do better.

std::array is just the built-in array type C++ should have but doesn't. This shouldn't be a library type, that's embarrassing.

std::deque looks like you're getting something like Rust's VecDeque but you aren't, it's a weird hybrid optimisation which presumably made sense on some 1980s hardware. I asked STL once to explain what it's even for and they didn't know. [[For reference STL is the name of the guy in charge of Microsoft's implementation of the C++ standard library, Microsoft also calls that library STL for reasons we needn't address]]

std::list is the extrusive doubly linked list. This type makes sense in a DSA class. Why is it in the C++ standard library? I dunno, maybe C++ is intended only as a teaching language?

std::forward_list is the extrusive singly linked list. You know, for a different seminar in that same DSA class. You might want the intrusive linked list, you don't want this.

std::map and std::set are probably red-black trees. OK, you might need those and for some reason not care about the details (which aren't specified here)

std::multimap and std::multiset are even less obviously useful. I have never seem them used in real software. Why are they in the standard library?

std::unordered_all_of_the_above_maps_and_sets look like the simplistic hash table you'd be shown in an intro DSA class either taught by somebody who doesn't know the subject well or aiming to cover the basics and get back to their research. This will perform poorly on any hardware with features like a cache.

The C++ stdlib carries broken garbage basically indefinitely. C++ doesn't have the same library stability promise that Rust has, but in practice stuff that nobody cares about is never removed.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#79

Post by janwas »

imtringued wrote: post #68
portly wrote: post #54

Earlier quoted context omitted.

Don't let the best be the enemy of the good. I got amazing performance for swapping for-loops with some simple SIMD patterns. Moreover. By doing this. I noticed that the codebase started to become better shaped for performance as well. By writing SIMD patterns, you get into the mindset of tight, hot loops.

The problem is that you're better off by defining SIMD friendly data structures and letting the compiler figure it out than by hand coding the actual SIMD operations. If you wanted to explicitly opt into bundling/batching of operations, you wouldn't actually want to define a fixed register size. You'd want a data type that represents an arbitrarily sized register and exposes some across batch operations. Then the com…

This works today :) Highway provides such an abstraction for arbitrary vector lengths and maps them to intrinsics. All on the library level, no need to wait years for compiler or language updates.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#80

Post by janwas »

jandrewrogers wrote: post #3

I have written a lot of SIMD for both x86 and ARM over many years and many microarchitectures. Every abstraction, including autovectorization, is universally pretty poor outside of narrow cases because they don’t (and mostly can’t) capture what is possible with intrinsics and their rather extreme variation across microarchitectures. If I want good results, I have to write intrinsics. No library can optimally generate…

In such discussions, whenever you mention abstractions are universally "pretty poor", to the extent anyone is listening, I think this hyperbole can do real damage. Maybe it prevents people from getting relevant performance gains, even if not 100% of the optimum, which is anyway unattainable. And what is the alternative? Not many projects can afford to hand write intrinsics for all platforms. And are you aware that Highway is basically a thin wrapper over intrinsics, which you can still drop down to where it helps?
Post reply on HN