Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C++26 Shipped a SIMD Library Nobody Asked For

lucisqr.substack.com

Post reply on HN
Print view View original

131–140 of 170 posts

Re: C++26 Shipped a SIMD Library Nobody Asked For

#131

Post by mpyne »

janwas wrote: post #75
mpyne wrote: post #9

Earlier quoted context omitted.

> I think a legitimate criticism is that it is unclear who std::simd is for. I think it's for people like me, who recognize that depending on the dataset that a lot of performance is left on the table for some datasets when you don't take advantage of SIMD, but are not interested in becoming experts on intrinsics for a multitude of processor combinations. Having a way to be able to say "flag bytes in this buffer matc…

Have you considered our Highway library? Runtime dispatch need not be a PITA :) It's basically portable intrinsics, and a much more complete set (>300) than the ~50 in std.

I hadn't but it would make sense for doing my own personal programming challenges.

Given the ongoing disasters around the software supply chains I've been fighting the creeping NPM-ism that people are trying to introduce to C++, where you just FetchContent 20 different libraries to build your own app upon.

I do use gtest, fmt and a few others though, so something as broadly used as Highway would probably be fine by that standard as well. But I'd still like it better if there was a Good Enough solution that was part of C++ stdlib to reduce the number of external integrations that are deemed required for a modern C++ program.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#132

Post by Validark »

If we want to improve cross-platform SIMD, in my opinion we should start by supporting more operations in LLVM IR. Like vector expansion (currently we only have expandload), runtime-known shuffle vectors, pdep/pext operations.

Also, let's stop with the "vector length agnostic" types being the sole option for SVE extensions. I'd rather write an optimized routine for a 16-byte machine I'm targeting and be able to upgrade it in 5 years than have "agnostic" code that wants to pretend like it would work amazingly on all platforms, but the machine I optimized it for is theoretical. I'm fine with recompiling my code, I do it every day. If I have an algorithm that's truly vector length agnostic, I can make the vector length a constant in my code that can change based on the compile target.

https://github.com/llvm/llvm-project/issues/113422

https://github.com/llvm/llvm-project/issues/172857

Re: C++26 Shipped a SIMD Library Nobody Asked For

#133

Post by camel-cdr »

Validark wrote: post #132

If we want to improve cross-platform SIMD, in my opinion we should start by supporting more operations in LLVM IR. Like vector expansion (currently we only have expandload), runtime-known shuffle vectors, pdep/pext operations. Also, let's stop with the "vector length agnostic" types being the sole option for SVE extensions. I'd rather write an optimized routine for a 16-byte machine I'm targeting and be able to upgra…

> Also, let's stop with the "vector length agnostic" types being the sole option for SVE extensions

They aren't, see the `arm_sve_vector_bits` attribute.

> I'm fine with recompiling my code, I do it every day

Then you can do that.

> If I have an algorithm that's truly vector length agnostic, I can make the vector length a constant in my code that can change based on the compile target.

You can do that, but why not simply write it in a vector-length-agnostic way?

IMO the better approach is to start thinking about SIMD optimizations in a VLA way, and specialize on the vector length, when that becomes advantageous. Doing it this way is better even if you end up not writing VLA code, because you though about the scalability problem.

Many libraries currently don't scale beyond 128-bit, not because they couldn't make efficient use of >128-bit, but because the library was architect around 128-bit and changing that amounts to almost a full rewrite. So now you are stuck wasting 3/4th of your ALUs running 128-bit SSE on Zen5.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#134

Post by sscaryterry »

fg137 wrote: post #104
sscaryterry wrote: post #87

Unnecessarily negative article. Lets not forget how awful C++98 was for years. Standardisation doesn't mean useful.

Hmm. I think you missed the point.

No, I didn't. The whole premise is contained in the title "Nobody asked for".

Re: C++26 Shipped a SIMD Library Nobody Asked For

#135

Post by janwas »

CoastalCoder wrote: post #82
janwas wrote: post #80

Earlier quoted context omitted.

In such discussions, whenever you mention abstractions are universally "pretty poor", to the extent anyone is listening, I think this hyperbole can do real damage. Maybe it prevents people from getting relevant performance gains, even if not 100% of the optimum, which is anyway unattainable. And what is the alternative? Not many projects can afford to hand write intrinsics for all platforms. And are you aware that Hi…

> 100% of the optimum, which is anyway unattainable. Can you expand on this? Sounds like an interesting discussion.

:) I figure there is always something left to improve. For some kernels which really want to keep 30+ live registers, the compiler might not do as good a job as careful manual tuning, so intrinsics can have a bit of a cost. But I also figure optimization time is limited, so better to get 90% of several kernels rather than one to 99%.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#136

Post by janwas »

mpyne wrote: post #131
janwas wrote: post #75

Earlier quoted context omitted.

Have you considered our Highway library? Runtime dispatch need not be a PITA :) It's basically portable intrinsics, and a much more complete set (>300) than the ~50 in std.

I hadn't but it would make sense for doing my own personal programming challenges. Given the ongoing disasters around the software supply chains I've been fighting the creeping NPM-ism that people are trying to introduce to C++, where you just FetchContent 20 different libraries to build your own app upon. I do use gtest, fmt and a few others though, so something as broadly used as Highway would probably be fine by t…

Fair point. If it helps, our security team has called Highway critical infrastructure and helped to harden the repo. The flip side of standardization is that it would be much harder and slower to add ops as the need arises, which we do regularly.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#137

Post by janwas »

fweimer wrote: post #122
janwas wrote: post #98

Earlier quoted context omitted.

Yes, the EMU128 target is scalar only, with for loops. This is a fun way to see how well autovectorization works, with the same source code. That works on any CPU. Curious which projects have such concerns, any link?

People reported challenges building V8 (whether upstream or the Node.js variant) on s390x with z13 support. I don't know if it was discussed on the porters mailing list because it's not public: https://groups.google.com/g/v8-s390-ports Elsewhere, some people interpreted https://github.com/google/highway/issues/1895 as meaning that Highway code does not work on z13 at all.

Thanks for sharing. The first link seems non public indeed. I can imagine there is some compile issue we could reasonably fix, with the help of someone who has Z13 access. Please encourage them to raise an issue. I will be back on May 26. After that, it should at least be able to use the scalar fallback. The issue with Z14 is that it lacks fp32 support. Would their usage be integer only?

Re: C++26 Shipped a SIMD Library Nobody Asked For

#138

Post by janwas »

duped wrote: post #129
janwas wrote: post #77

Earlier quoted context omitted.

:) I agree a tutorial would be helpful. We are working on one with Fastcode.

A manual is not a tutorial, and having AI anywhere near this task is actively harmful. Please do not build this.

?? Where did you see mention of AI?

Re: C++26 Shipped a SIMD Library Nobody Asked For

#139

Post by lostglass »

jcranmer wrote: post #119
Sharlin wrote: post #70

The point about the optimizer only seeing "opaque templates and function calls" makes little sense. First off, templates are the opposite of opaque due to the fundamental requirement that the implementation be visible to every translation unit using a template. This makes any function calls trivially inlinable. Second, and the reason for the above requirement, templates are compiled by monomorphization – making a dis…

As a compiler guy, the complaint about "opaque templates and function calls" to me raises serious doubts that the author has any idea what they're talking about. std::simd is designed to be akin to taking vector operations as intrinsics on and similar types and wrapping them in a more C++ dialect than bare compiler intrinsics (and then a second layer on top of that to make things somewhat more portable). So the imple…

So I've got a foot in each camp, I think you're just using different languages - you guys mean different things with the same words.

You can't claim he doesn't know what he's talking about with a single point he may have gotten run, the makes a tonne of valid points, especially around the existing problems of C++ that this library doesn't help with.

In addition to that, he's not wrong about this library from a user perspective. I can't use this. I wrote something very similar back in 2016 - at the time it served my needs but now it's hilariously outdated.

Re: C++26 Shipped a SIMD Library Nobody Asked For

#140

Post by feelamee »

jcranmer wrote: post #124
feelamee wrote: post #123

Earlier quoted context omitted.

are u carefully read paper[1]? It doesn't require to reimplement it... > Our proposal is inspired by and extends the dense BLAS interface. A natural implementation might look like this: > 1. wrap an existing C or Fortran BLAS library, [1] https://www.open-std.org/jtc1/sc22/wg21/docs/papers/2023/p16...

Have you read the entire paper, and not just skimmed the front matter? The interface is a generic template approach, which can work on any element type T, not just float/double/complex /complex , but custom types like bigint or rational or random_custom_finite_field. Or integration with units libraries (there's another dumpster fire coming down the line...). Your BLAS library will provide you just the four basic elem…

> Your BLAS library will provide you just the four basic element types, ..., and you still need fallback logic anyways to handle the other types.

so, your problem with it is that it does all you want (e.g. LAPACK bindings) AND give extra features??

> so it takes a decent amount of dispatch logic to convert the template interface to the actual library calls

I can't estimate how much this degrades performance. But, it feels very low overhead compared to the calculation itself (and probably should be resolved at compile time)

Post reply on HN