Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Using SIMD for Parallel Processing in Rust

nrempel.com

Post reply on HN
Print view View original

21–30 of 43 posts

Re: Using SIMD for Parallel Processing in Rust

#21

Post by camel-cdr »

IshKebab wrote: post #14

Minor nit: RISC-V Vector isn't SIMD. It's actually like ARM's Scalable Vector Extension. Unlike traditional SIMD the code is agnostic to the register width and different hardware can run the same code with different widths. There is also a traditional SIMD extension (P I think?) but it isn't finished. Most focus has been on the vector extension. I am wondering how and if Rust will support these vector processing exte…

> RISC-V Vector isn't SIMD

Isn't SIMD a subset of vector processors?

To that matter, can anybody here provide a proper and useful distinction between the two, that is SIMD and vector ISAs?

You imply it's because it's vector length agnostic, but you could take e.g. the SSE encoding, and apart from a few instructions, make it operate on SIMD registers of any length. Wouldn't that also be vector length agnostic, as long as software can query the vector length? I think most people wouldn't call this a vector ISA, and how is this substantially different from dispatching to different implementations for SSE AVX and AVX512?

I've also seen people say it's about the predication, which would make AVX512 a vector isa.

I've seen others say it's about resource usage and vector chaining, but that is just an implementation detail and can be used or not used on traditional SIMD ISAs to the same extend as on vector ISAs.

Re: Using SIMD for Parallel Processing in Rust

#22

Post by ww520 »

Zig actually has a very nice abstraction for SIMD in the form of vector programming. The size of the vector is agnostic to the underlying cpu architecture. The compiler or LLVM will generate code for using SIMD128, 256, or 512 registers. And you are just programming straight vectors.

Re: Using SIMD for Parallel Processing in Rust

#23

Post by neonsunset »

runevault wrote: post #20
neonsunset wrote: post #19

Earlier quoted context omitted.

What kind of private internals do you have in mind? You absolutely can hand-roll your own comparison routine, just hard to beat existing implementation esp. once you start considering culture-sensitive comparison (which may defer to e.g. ICU). There are no private SIMD APIs save for sequence comparison intrisic for unrolling against known lengths which JIT/ILC does for spans and strings.

IIRC (Been a month or so since I looked into it) I couldn't access the underlying array in a way SIMD liked I think? If you look at how they did it inside the actual string class it uses those private properties of the string that are only available internally to guarantee you don't change the string data if memory serves.

String can provide you a `ReadOnlySpan`, out of which you can either take `ref readonly char` "byref" pointer, which all vectors work with, or you can use the unsafe variant and make this byref mutable (just don't write to it) with `Unsafe.AsRef`.

Because pretty much every type that has linear memory can be represented as span, it means that every span is amenable to pointer (byref) arithmetics which you then use to write a SIMD routine. e.g.:

    var text = "Hello, World! Hello, World!";
    var span = MemoryMarshal.Cast(text);
    ref readonly var ptr = ref span[0];

    var chunk = Vector128.LoadUnsafe(in ptr);
    var needle = Vector128.Create((ushort)',');
    var comparison = Vector128.Equals(chunk, needle);
    var offset = uint.TrailingZeroCount(comparison.ExtractMostSignificantBits());

    Console.WriteLine(text[..(int)offset]);
If you have doubts regarding codegen quality, take a look at: https://godbolt.org/z/b97zjfTP7 The above vector API calls are lowered to lines 17-22.

Re: Using SIMD for Parallel Processing in Rust

#24

Post by runevault »

neonsunset wrote: post #23
runevault wrote: post #20

Earlier quoted context omitted.

IIRC (Been a month or so since I looked into it) I couldn't access the underlying array in a way SIMD liked I think? If you look at how they did it inside the actual string class it uses those private properties of the string that are only available internally to guarantee you don't change the string data if memory serves.

String can provide you a `ReadOnlySpan `, out of which you can either take `ref readonly char` "byref" pointer, which all vectors work with, or you can use the unsafe variant and make this byref mutable (just don't write to it) with `Unsafe.AsRef`. Because pretty much every type that has linear memory can be represented as span, it means that every span is amenable to pointer (byref) arithmetics which you then use to…

Oh interesting, I'll have to give that a try then. My concern was avoiding a reallocation by doing it another way, but if the readonly span works I can see how it would get you there. I need to see if I still have that project to test it out, appreciate the heads up. SIMD is something I really want to get better with.

Re: Using SIMD for Parallel Processing in Rust

#25

Post by neonsunset »

runevault wrote: post #24
neonsunset wrote: post #23

Earlier quoted context omitted.

String can provide you a `ReadOnlySpan `, out of which you can either take `ref readonly char` "byref" pointer, which all vectors work with, or you can use the unsafe variant and make this byref mutable (just don't write to it) with `Unsafe.AsRef`. Because pretty much every type that has linear memory can be represented as span, it means that every span is amenable to pointer (byref) arithmetics which you then use to…

Oh interesting, I'll have to give that a try then. My concern was avoiding a reallocation by doing it another way, but if the readonly span works I can see how it would get you there. I need to see if I still have that project to test it out, appreciate the heads up. SIMD is something I really want to get better with.

If you go through the guide at the first link, it will pretty much set you up with the basics to work on vectorization, and once done, you can look at what CoreLib does as a reference (just keep in mind it tries to squeeze all the performance for short lengths too, so the tail/head scalar handlers and dispatch can be high-effort, more so than you may care about). The point behind the way .NET does it is to have the same API exposed to external consumers as the one CoreLib uses itself, which is why I was surprised by your initial statement.

No offense taken, just clarifying, SIMD can seem daunting especially if you look at intrinsics in C/C++, and I hope the approach in C# will popularize it. Good luck with your experiments!

Re: Using SIMD for Parallel Processing in Rust

#26

Post by runevault »

neonsunset wrote: post #25
runevault wrote: post #24

Earlier quoted context omitted.

Oh interesting, I'll have to give that a try then. My concern was avoiding a reallocation by doing it another way, but if the readonly span works I can see how it would get you there. I need to see if I still have that project to test it out, appreciate the heads up. SIMD is something I really want to get better with.

If you go through the guide at the first link, it will pretty much set you up with the basics to work on vectorization, and once done, you can look at what CoreLib does as a reference (just keep in mind it tries to squeeze all the performance for short lengths too, so the tail/head scalar handlers and dispatch can be high-effort, more so than you may care about). The point behind the way .NET does it is to have the s…

I appreciate you taking the time to talk me through this, SIMD has been an interest of mine for a while. I ran into issues and then when I went and looked at how the actual string class did it I stopped since they were doing tricks that required said access to the internal data. But this gives me a path to explore. I was already planning on looking at the links you supplied.

Thank you again.

Re: Using SIMD for Parallel Processing in Rust

#27

Post by Calavar »

runevault wrote: post #16
Calavar wrote: post #12

Earlier quoted context omitted.

What would native SIMD support entail in a language without first party JIT or AOT compilation?

At some point bytecode still turns into CPU instructions, so if you added syntax or special functions that went to parts of the interpreter that are SIMD you could certainly add it to a purely interpreted language.

If we're talking low level SIMD, like opcode level, I'm really struggling to see the use case for interpreted bytecode. The cost of type checking operands to dynamically dispatch down a SIMD path would almost certainly outweigh the savings of the SIMD path itself.

JIT is different because in function-level JIT, you can check types just once at the opening of the function, then you stay on the SIMD happy path for the rest of the function. And in AOT, you may able to elide the checks entirely.

There is certainly a space for higher level SIMD functionality. Numpy is one example.

Re: Using SIMD for Parallel Processing in Rust

#28

Post by devit »

nbrempel wrote: post #5

Thanks for reading everyone. I’ve gotten some feedback over on Reddit as well that the example is not effectively showing the benefits of SIMD. I plan on revising this. One of my goals of writing these articles is to learn so feedback is more than welcome!

Are you really writing them?

Seems written by an LLM for the most part.

Re: Using SIMD for Parallel Processing in Rust

#29

Post by dzaima »

nbrempel wrote: post #5

Thanks for reading everyone. I’ve gotten some feedback over on Reddit as well that the example is not effectively showing the benefits of SIMD. I plan on revising this. One of my goals of writing these articles is to learn so feedback is more than welcome!

What's fun is that, as the use of SIMD in your example is useless, LLVM correctly completely removes it, and makes your "neon" and "fallback" versions exactly the same - without any SIMD (compiler explorer: https://godbolt.org/z/YWoMGoaxT).

As an additional note, aarch64 always has NEON (similar to how x86-64 always has SSE2; extensions useful to dispatch would be SVE on aarch64 and AVX/AVX2/AVX-512 on x86-64), so no point dynamically checking for it.

Re: Using SIMD for Parallel Processing in Rust

#30

Post by jvanderbot »

ww520 wrote: post #22

Zig actually has a very nice abstraction for SIMD in the form of vector programming. The size of the vector is agnostic to the underlying cpu architecture. The compiler or LLVM will generate code for using SIMD128, 256, or 512 registers. And you are just programming straight vectors.

Yeah, the article overlooked library support for SIMD. nalgebra had a decent writeup on their ability to squeeze out autovectorization for their vector and matrix types.
Post reply on HN