Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C++26 Shipped a SIMD Library Nobody Asked For

lucisqr.substack.com

Post reply on HN
Print view View original

51–60 of 170 posts

Re: C++26 Shipped a SIMD Library Nobody Asked For

#51

Post by IshKebab »

stephbook wrote: post #34
AlotOfReading wrote: post #12

Earlier quoted context omitted.

Trying to abstract over SVE with a SIMD library is a bit of a fool's errand. The intended programming model is just too different from traditional ISAs, and there are algorithms that are nearly impossible to write efficiently for it. All the ones I've seen wrap it up as a bastardized fixed length ISA, and even ARM's own guidance basically recommends that approach. Frankly, the length agnostic stuff is a mistake that…

I'm no C++ dev, but as an outsider, it sure reads like the whole "int is variable length" mistake again.

In a way it's worse because at least with int you're not really expecting to run the same binary on architectures with different int lengths, and also for several decades there have only been two realistic options (32 or 64), which makes it easy to deal with.

With RVV (and SVE I assume) there are a wider range of realistic options - at least 128, 256 and 512. The RVV spec allows up to 65536! Also it's totally reasonable to want a single binary to work with all of them so then you're into compiling parts of your code multiple times with runtime dispatch which is a right pain.

I haven't looked into how Highway does it but I don't really know you you write length-agnostic code in high level languages. It's easy in assembly, but it sucks if you have to do it in assembly.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#52

Post by camel-cdr »

AlotOfReading wrote: post #43
camel-cdr wrote: post #29

Earlier quoted context omitted.

> Trying to abstract over SVE with a SIMD library is a bit of a fool's errand It reallt isn't. You just make the default SIMD-width agnostic and anything less portable opt-in. You can still specialize for a specific width pn scalabe vector ISAs. > The intended programming model is just too different from traditional ISAs, and there are algorithms that are nearly impossible to write efficiently for it. Such as? > All…

Such as? I have a database that has big columns that get functions applied to them to compute the result set. This is a perfect case for length agnostic instructions, except out ends up horribly memory bound. A nice optimization is to only compute those lanes containing rows that might actually be in the result set by keeping track of a sparse record that depends on the lane size. But the cnt instructions are optiona…

CNT and CNTP don't seem to be optional for SVE, from what I found. (unless you mean HISTCNT)

It seems to me like you want tp use CNTP on a bitset that tells you, which rows are relevant, skipping them if CNT is 0? Is that what you where describing?

Re: C++26 Shipped a SIMD Library Nobody Asked For

#53

Post by zombot »

The article's point in a nutshell:

> The problem is that std::simd in 2026 is the 2012 solution arriving after the world moved on. The committee spent a decade polishing a library-based approach while compilers solved the easy cases automatically and ISPC solved the hard cases with language-level support.

I find it interesting that the C++ committee would make that kind of mistake. Shouldn't they know better?

Re: C++26 Shipped a SIMD Library Nobody Asked For

#54

Post by portly »

mgaunard wrote: post #6
jandrewrogers wrote: post #3

I have written a lot of SIMD for both x86 and ARM over many years and many microarchitectures. Every abstraction, including autovectorization, is universally pretty poor outside of narrow cases because they don’t (and mostly can’t) capture what is possible with intrinsics and their rather extreme variation across microarchitectures. If I want good results, I have to write intrinsics. No library can optimally generate…

For me the main issue is that if you're serious about SIMD, you need to use a state-of-the-art library and can't rely on some standard library whose quality is variable, unreliable, and which is by design always behind.

Don't let the best be the enemy of the good. I got amazing performance for swapping for-loops with some simple SIMD patterns. Moreover. By doing this. I noticed that the codebase started to become better shaped for performance as well. By writing SIMD patterns, you get into the mindset of tight, hot loops.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#56

Post by pwdisswordfishq »

jcranmer wrote: post #38

If you thought std::simd was a library nobody asked for, just wait until you hear about . I feel like half the people looking forward to that think they're just going to get standard C++ bindings to LAPACK, when instead they're probably going to get an unoptimized, slapdash implementation of LAPACK written by people who aren't good at BLAS. As for SIMD itself, designing a good SIMD library is difficult because there…

Just wait until you hear about std::hive.

The work of one obsessive author, who never gave a good explanation for why the thing needed to be in the standard library instead of an external one. The committee was apathetic about the proposal and kept bringing up various trivial issues, in a clear attempt to stall him, but he refused to take the hint. So eventually they relented. Outside coverage I have seen so far seems to be to the tune of "WTF is this weird thing?" and quickly glosses over it.

I wonder if it's going to end up like the export keyword.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#57

Post by jandrewrogers »

camel-cdr wrote: post #45
jandrewrogers wrote: post #41

Earlier quoted context omitted.

Almost literally what I stated. Consider a row in Postgres table or similar. Convert the entire WHERE clause across all columns in that table into a very short sequence of SIMD instructions against the same memory. All of the columns, regardless of type, are evaluated simultaneously using SIMD. For many complex constraints you can match rows in single digit clock cycles even across many unrelated types. This is much…

OK, I thought it couldn't be that, because that should be doable with std::simd or a SIMD abstraction. Well, unless you JIT it, in which case intrinsics wouldn't help either. > You can match search patterns across a random dozen columns across a schema of hundreds of columns at essentially full memory bandwidth Do I underatand it correctly, that this would only work, if you have multiple of the same comparisons (e.g.…

Every column has its own independent constraint: equality, order, range intersection, bit sets, etc that is evaluated concurrently in single operations. Independent per column in parallel. It does require handling the representation of columns to enable it but that isn’t onerous in practice.

It isn’t intuitive but it is one of those things that is obvious in hindsight once you see how it works. The gap is that people struggle to understand how to make this something SIMD native, especially in high-performance systems.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#58

Post by raverbashing »

sigh

C++ sits on that weird abstraction level where it wants to be a higher level language but it keeps grinding their gears on stuff like pointer sizes, pointer arithmetic or vector sizes and at the same time wants to keep being C compatible and needs that interface with the lower level world

Now compare with how numpy does things: you care about the data size but not the implementation.

Still, I didn't expect less (of a crap fest) from the C++ committee as presented here

Re: C++26 Shipped a SIMD Library Nobody Asked For

#59

Post by ozgrakkurt »

Just write inline asm for x86 and aarch64 (if you care about that) and not care about the rest. Is it even useful to do simd on other processors?

Compiler optimizing even the code around the simd code based on the semantics of arithmetic or other things sounds silly after writing some of this kind of code

Re: C++26 Shipped a SIMD Library Nobody Asked For

#60

Post by camel-cdr »

ozgrakkurt wrote: post #59

Just write inline asm for x86 and aarch64 (if you care about that) and not care about the rest. Is it even useful to do simd on other processors? Compiler optimizing even the code around the simd code based on the semantics of arithmetic or other things sounds silly after writing some of this kind of code

So you "just" write 4 assembly implementations?
Post reply on HN