Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C++26 Shipped a SIMD Library Nobody Asked For

lucisqr.substack.com

Post reply on HN
Print view View original

121–130 of 170 posts

Re: C++26 Shipped a SIMD Library Nobody Asked For

#121

Post by StilesCrisis »

raverbashing wrote: post #116
StilesCrisis wrote: post #115

> When Google needed portable SIMD for production image and video codecs, they built Highway — not std::simd. Sure, they left the committee years ago. I am not trying to claim any sort of direct causality, but it sure seems like this is a case where Google's presence on the committee might have prevented shipping boondoggles like this. Modules is another case where I think Google's feedback might have been able to st…

They probably left the committee because it keeps going in circles rather than solving the issues of the language

Yes. The straw that broke the camel's back was the complete refusal to break ABI, locking in bad implementations forever. e.g. unordered_map dramatically underperforms when compared against modern swiss tables, but the committee won't do anything about it. Not to mention the committee's head-in-the-sand policy-based approaches to safety, vs Google's much broader-scoped Carbon effort.

More context: https://github.com/carbon-language/carbon-lang/blob/trunk/do...

Re: C++26 Shipped a SIMD Library Nobody Asked For

#122

Post by fweimer »

janwas wrote: post #98
fweimer wrote: post #85

Earlier quoted context omitted.

Does it have fallback paths for everything, though? Scalar if necessary? Projects that depend on Highway drop support for CPUs not listed in the Highway documentation, saying that they can't support these CPUs because they are incompatible with Highway: https://google.github.io/highway/en/master/README.html#curre... Are these projects somehow mistaken?

Yes, the EMU128 target is scalar only, with for loops. This is a fun way to see how well autovectorization works, with the same source code. That works on any CPU. Curious which projects have such concerns, any link?

People reported challenges building V8 (whether upstream or the Node.js variant) on s390x with z13 support. I don't know if it was discussed on the porters mailing list because it's not public: https://groups.google.com/g/v8-s390-ports

Elsewhere, some people interpreted https://github.com/google/highway/issues/1895 as meaning that Highway code does not work on z13 at all.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#123

Post by feelamee »

jcranmer wrote: post #38

If you thought std::simd was a library nobody asked for, just wait until you hear about . I feel like half the people looking forward to that think they're just going to get standard C++ bindings to LAPACK, when instead they're probably going to get an unoptimized, slapdash implementation of LAPACK written by people who aren't good at BLAS. As for SIMD itself, designing a good SIMD library is difficult because there…

are u carefully read paper[1]?

It doesn't require to reimplement it...

> Our proposal is inspired by and extends the dense BLAS interface. A natural implementation might look like this:

> 1. wrap an existing C or Fortran BLAS library,

[1] https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2023/p16...

Re: C++26 Shipped a SIMD Library Nobody Asked For

#124

Post by jcranmer »

feelamee wrote: post #123
jcranmer wrote: post #38

If you thought std::simd was a library nobody asked for, just wait until you hear about . I feel like half the people looking forward to that think they're just going to get standard C++ bindings to LAPACK, when instead they're probably going to get an unoptimized, slapdash implementation of LAPACK written by people who aren't good at BLAS. As for SIMD itself, designing a good SIMD library is difficult because there…

are u carefully read paper[1]? It doesn't require to reimplement it... > Our proposal is inspired by and extends the dense BLAS interface. A natural implementation might look like this: > 1. wrap an existing C or Fortran BLAS library, [1] https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2023/p16...

Have you read the entire paper, and not just skimmed the front matter?

The interface is a generic template approach, which can work on any element type T, not just float/double/complex/complex, but custom types like bigint or rational or random_custom_finite_field. Or integration with units libraries (there's another dumpster fire coming down the line...). Your BLAS library will provide you just the four basic element types, so it takes a decent amount of dispatch logic to convert the template interface to the actual library calls, and you still need fallback logic anyways to handle the other types.

But the library is also not designed in a way to facilitate that kind of dispatch logic (std::simd is, which accounts for a not insubstantial portion of its complexity). Which is on top of the difficulty of linking to one of various BLAS implementations as a standard library. So it's a design that's all but guaranteed to let you link against an existing BLAS implementation, and indeed, carefully reading the rest of the section you wrote makes it clear that it's not a goal of the paper proposal to have implementations do that.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#125

Post by saagarjha »

fulafel wrote: post #66
jandrewrogers wrote: post #3

I have written a lot of SIMD for both x86 and ARM over many years and many microarchitectures. Every abstraction, including autovectorization, is universally pretty poor outside of narrow cases because they don’t (and mostly can’t) capture what is possible with intrinsics and their rather extreme variation across microarchitectures. If I want good results, I have to write intrinsics. No library can optimally generate…

Autovectorisation is the main way SIMD hardware gets put into use, whether you think it's pretty poor or not. SIMD came to mainstream in 1995 Pentium MMX and has been proven rather difficult for compilers to target, but after 30+ years is doing a bit better despite PLT conspiring against it. (see eg CUDA, Futhark etc)

I think the main way SIMD hardware gets put to use is probably memcpy.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#126

Post by jcranmer »

zombot wrote: post #53

The article's point in a nutshell: > The problem is that std::simd in 2026 is the 2012 solution arriving after the world moved on. The committee spent a decade polishing a library-based approach while compilers solved the easy cases automatically and ISPC solved the hard cases with language-level support. I find it interesting that the C++ committee would make that kind of mistake. Shouldn't they know better?

The C++ standards committee is pretty damn dysfunctional at this point for a variety of reasons.

Only like 10% of the committee are actually responsible for an implementation in some manner; the vast majority are users, often looking to get their feature into the standard. This also means that only a tiny minority of the committee actually understands things like the difference between a prototype hack and a proper implementation. I get the sense that it's extremely bad on the library front--all of the standard library implementors I know are basically pleading "please stop adding new features, we want time to catch up."

One of the big issues with library features is that library vendors can't just copy-paste existing implementations for licensing reasons, so they have to reimplement it largely from scratch, and they people doing so may not necessarily be skilled in that particular domain. On top of that, standard libraries are much more sensitive to ABI breaks than other libraries are, so a bad design gets ossified to a much worse degree than regular libraries. The best examples of baked-in bad implementations are std::unordered_map and std::regex, but honestly even std::unique_ptr has similar ABI-unfixable issues (it's not a pointer for ABI calling conventions). Yet you still see people cheer on additions to the standard library because obviously those people are going to make existing implementations better.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#127

Post by adrian_b »

whizzter wrote: post #93

Agreed, fixed with vectors needs to be a language feature, better compile times and would solve issues for most people. Personally, I think that like Clang way to adding GLSL like vectors and semantics would've gone a long way. SVE might be an elegant design, but in reality there are probably a multiple factor of game and other 3d code being written that needs vectors compared to other fields, and there limited vecto…

Intel has been forced to reintroduce 512-bit vectors in the mainstream, because of the competition from AMD.

Starting with the Intel Nova Lake CPUs, around the end of this year, all future AMD and Intel CPUs will provide 512-bit vectors, like also the current AMD Zen 5 and Zen 4 CPUs.

The 512-bit vector length is more convenient than other lengths, because on the AMD and Intel CPUs it coincides with the length of a cache line. Because of this, it is easier to optimize simultaneously for the best cache usage.

For GPUs, which favor throughput over latency, 1024-bit and 2048-bit vector register widths are frequently used. For CPUs it is unlikely that widths greater than 512-bit would be useful, as the vector operations that should be done on CPUs are those for which the high latency of using a GPU is undesirable.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#128

Post by meling »

camel-cdr wrote: post #92
meling wrote: post #67

Curious if people here have looked at the upcoming SIMD support in Go: https://go.dev/doc/go1.26#simd Currently experimental, but looks like the first Intel arch will arrive in the next release in about 3 months. They are also going to support a portable layer. Wondering what people here think about the approach the Go team is taking; I think they would appreciate more eyeballs on their design. (I’m not competent in…

Looks like that isn't a portable SIMD abstraction, but more similar to adding architecture-specific SIMD intrinsics support to go, with nicer syntax.

Sorry, I didn’t explicitly link to the issue for the portal layer.

Here is the issue discussing the portal simd package: https://github.com/golang/go/issues/78902

Re: C++26 Shipped a SIMD Library Nobody Asked For

#129

Post by duped »

janwas wrote: post #77
duped wrote: post #21

Earlier quoted context omitted.

> it's a PITA to differentiate the utility of hundreds of different vpaddcfoolol instructions This is one complaint I toss back at Intel and AMD. If an instruction/intrinsic is universally worse than the P90/P95/P99 use case where it's going to be used to another set of instrinsics, then it shouldn't exist. Stop wasting the die space and instruction decode on it, if not only the developer time wasted finding out that…

:) I agree a tutorial would be helpful. We are working on one with Fastcode.

A manual is not a tutorial, and having AI anywhere near this task is actively harmful. Please do not build this.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#130

Post by camel-cdr »

adrian_b wrote: post #127
whizzter wrote: post #93

Agreed, fixed with vectors needs to be a language feature, better compile times and would solve issues for most people. Personally, I think that like Clang way to adding GLSL like vectors and semantics would've gone a long way. SVE might be an elegant design, but in reality there are probably a multiple factor of game and other 3d code being written that needs vectors compared to other fields, and there limited vecto…

Intel has been forced to reintroduce 512-bit vectors in the mainstream, because of the competition from AMD. Starting with the Intel Nova Lake CPUs, around the end of this year, all future AMD and Intel CPUs will provide 512-bit vectors, like also the current AMD Zen 5 and Zen 4 CPUs. The 512-bit vector length is more convenient than other lengths, because on the AMD and Intel CPUs it coincides with the length of a c…

greater then 512-bit SIMD isn't currently and in the near future relevant for regular general purpose processors.

But for smaller more specialized CPUs in embedded or automotive usecases you can get more parallel compute, while keeping the software model simpler than having to dispatch to a GPU.

Specifically a design like https://saturn-vectors.org/#_short_vector_execution, which like to use 2x or 4x wider vectors that the datapath length for more efficient chaining. I quite like that design, because you can get high utilization and limited out-of-order execution without vector register renaming.

Post reply on HN