Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust SIMD on the GPU

vectorware.com

Post reply on HN
Print view View original

91–100 of 129 posts

Re: Rust SIMD on the GPU

#91

Post by derefr »

dwattttt wrote: post #82
derefr wrote: post #66

Earlier quoted context omitted.

Only if the end-user is the one compiling the software, on the same very system they'll be running it on. Which is true of GPU shader kernels, due to how GPU drivers work; but isn't generally true of CPU object code (unless you're on Gentoo.) What you'd actually want is a matrix of variant implementations burned into the binary, with runtime (or process-boot-time) hardware detection that swaps symbols out to point to…

If you want the library to perform that selection, you also need the "correct" / most efficient implementation to be independent of your workload. I'm not that familiar with SIMD performance characteristics, but I wouldn't be surprised if that's not always the case.

From how I understand it, there'd likely only be a single SIMD function impl per uarch that'd actually be fully legally executable without hitting undefined instructions. Plus increasingly-more-generic function impls compiled for lower and lower common-denominator subsets of SIMD functionality. (Ultimately grounding in a non-SIMD impl.)

If that's the case, then the selection logic would be trivial: figure out the full hierarchical ID of the uarch you're running on, then search for the longest prefix match in the table of available impls.

If things work more like you're imagining, though, then I suppose the process-boot impl-selector would narrow down the impl matrix to just the subset that are legal on the running uarch; pick one arbitrarily to be active at first; and then wrap the calls in a handler that gradually re-works the called function in a way reminiscent of a profile-guided JIT, but without the need to actually synthesize any code at runtime — instead, it'd just be a multi-armed bandit passing-through-to and re-ranking competitor impls, with decreasing sampling of the non-first-ranked impls as confidence-in-score-separation increases.

Re: Rust SIMD on the GPU

#92

Post by pjmlp »

zamadatix wrote: post #57
camel-cdr wrote: post #37

I love how ever example of portable SIMD isn't portable. They specifies a constant SIMD width so it's non-portable. Well, not performance portable, but why are we using SIMD again?

It should really be read/advertised as "portabler SIMD". It beats hoping the compiler autovectorizes everything well forever or writing architecture specific code manually again but is going to compromise on average performance vs platform specific SIMD.

.NET and Java have three levels of SIMD support, Go's ongoing efforts, and does the upcoming C++ standard.

Autovectorization, depending on compiler's cleverness, really portable SIMD operations, and then the CPU specific SIMD ones.

So this should be perfectly doable in crate that advertises as portable, while leaving the non portable stuff to another crate.

Re: Rust SIMD on the GPU

#93

Post by pjmlp »

krapht wrote: post #65
IshKebab wrote: post #61

Earlier quoted context omitted.

Sure but there's no real way to use that in a portable way, at least not a way that maximises performance on every CPU you run it on. That's pretty much impossible at the moment.

Which is why I've never quite understood the appeal of portable SIMD libraries for performance-critical code. If I'm explicitly writing SIMD rather than relying on the auto-vectorizer, it's usually because I want access to the particular capabilities of the target ISA. For many problems, choosing the right instruction or instruction sequence makes a large difference. Portable SIMD abstractions necessarily expose some…

Because usually they achieve a very good middle ground, they are useful for when autovectorization isn't good enough, and it is possible to give a little help to the compiler.

There are many ways that performance matters without trying to win a F1 race.

Go isn't alone, .NET, Java have similar portable libraries, and C++ is in the process of getting one.

Re: Rust SIMD on the GPU

#94

Post by pjmlp »

vatsachak wrote: post #81
frollogaston wrote: post #76

I've noticed a lot of articles about SIMD on the HN front page. That's cool, but just wondering, is there some reason this is more in focus lately?

SIMD is actually underrated still. Programmers should always be thinking about it. It's a free 4x in a lot of cases

The main problem with SIMD is it is a complex subject, even if not that good, autovectorization wins over what most common devs know about SIMD.

Well now you could in theory AI generate SIMD, which will be vibe coded, as those devs have no idea of its correctness.

Re: Rust SIMD on the GPU

#95

Post by adgjlsfhk1 »

derefr wrote: post #66
Groxx wrote: post #64

Earlier quoted context omitted.

It'd need some kind of compile-time hardware-feature-detection, yea? That seems probably feasible since proc macros can do essentially anything they like (worryingly).

Only if the end-user is the one compiling the software, on the same very system they'll be running it on. Which is true of GPU shader kernels, due to how GPU drivers work; but isn't generally true of CPU object code (unless you're on Gentoo.) What you'd actually want is a matrix of variant implementations burned into the binary, with runtime (or process-boot-time) hardware detection that swaps symbols out to point to…

This is one of the nice things about JIT languages. you defer the compiler time decisions to runtime and this get to choose based on what the user has

Re: Rust SIMD on the GPU

#96

Post by exDM69 »

camel-cdr wrote: post #37

I love how ever example of portable SIMD isn't portable. They specifies a constant SIMD width so it's non-portable. Well, not performance portable, but why are we using SIMD again?

> They specifies a constant SIMD width so it's non-portable.

This is incorrect, you can use vectors wider than native SIMD width and the compiler will break them down to register size of the target cpu.

In fact it's sometimes better to used wider than native width, in some applications I see 20% better throughput with f32x16 (512 bits) on an AVX2 CPU (256 bits). It is kinda like loop unrolling it.

Re: Rust SIMD on the GPU

#97

Post by exDM69 »

MomsAVoxell wrote: post #47
camel-cdr wrote: post #37

I love how ever example of portable SIMD isn't portable. They specifies a constant SIMD width so it's non-portable. Well, not performance portable, but why are we using SIMD again?

Why should it be portable? Honest question. SIMD seems to me, to be very platform specific. Maybe there are times one SIMD unit is not anothers' SIMD unit?

Counter question: why shouldn't it be portable?

It's definitely a 80% solution where you occasionally need to drop down to intrinsics (at zero runtime perf cost) for CPU specific instructions.

But just having vector types, arithmetic, swizzling, loads and stores will go a long way for basic tasks.

And with generics you can write code that is type and width agnostic. No need to rewrite your code of you want to go from SSE to AVX512, just change from f32x4 to f32x16 (or use generics) and you are done.

Re: Rust SIMD on the GPU

#98

Post by camel-cdr »

exDM69 wrote: post #96
camel-cdr wrote: post #37

I love how ever example of portable SIMD isn't portable. They specifies a constant SIMD width so it's non-portable. Well, not performance portable, but why are we using SIMD again?

> They specifies a constant SIMD width so it's non-portable. This is incorrect, you can use vectors wider than native SIMD width and the compiler will break them down to register size of the target cpu. In fact it's sometimes better to used wider than native width, in some applications I see 20% better throughput with f32x16 (512 bits) on an AVX2 CPU (256 bits). It is kinda like loop unrolling it.

Except you can't use this in actual code, because either, as is the case in this example with f32x32, you run out of registers and spill all over the place. Or you aren't using your full vector register or could've gotten better performance by "unrolling" more often for the larger vectors.

If you use f32x16 (the avx-512 wisth), SSE now effectively has 4 registers to work with and will spill when doing anything beyond the most simple stuff.

The default should imo be relative to the native register width, so you can do 1x, 2x or sometimes 4x the native width, depensing on your register preasure.

Re: Rust SIMD on the GPU

#99

Post by stymaar »

LoganDark wrote: post #78
jonkoops wrote: post #45

Earlier quoted context omitted.

Pretty common for Rust to cook things in nightly for a very long time; I wouldn't consider it a bad thing, tbh.

It's been annoying to me as an end user that so many basic things require nightly. I use nightly as my main toolchain, but enabling unstable features makes a project nightly-only, which is undesired for crates that don't already revolve around the unstable feature. I most often encounter unstable features when I reach for a basic common-sense utility method and discover that it's not stable. Like just earlier today I…

> It's been annoying to me as an end user that so many basic things require nightly

It used to be the case a decade ago, but now I wouldn't agree that any "basic" things require nightly (I wouldn't call portable SIMD "basic" at all for instance).

> Like just earlier today I would have reached for bool::toggle which not only is unstable, but is also newly added as of like a month ago!

This is very likely not the kind of feature that will stay on nightly for a long time, but is instead one of the many convenience feature that land on stable every release. The 6-weeks release cadence with beta in between means there's always at least 6 weeks and up to 3 months between the time a feature land on nightly and the day it reaches stable, even if the feature is as consensual as this one.

> And now that IntelliJ-Rust is proprietary, I can't even make a feature request anymore for the ability to exclude unstable features from the autocomplete.

Can't you tell it to use stable as the default target, and use nightly manually in cargo?

Re: Rust SIMD on the GPU

#100

Post by LoganDark »

stymaar wrote: post #99
LoganDark wrote: post #78

Earlier quoted context omitted.

It's been annoying to me as an end user that so many basic things require nightly. I use nightly as my main toolchain, but enabling unstable features makes a project nightly-only, which is undesired for crates that don't already revolve around the unstable feature. I most often encounter unstable features when I reach for a basic common-sense utility method and discover that it's not stable. Like just earlier today I…

> It's been annoying to me as an end user that so many basic things require nightly It used to be the case a decade ago, but now I wouldn't agree that any "basic" things require nightly (I wouldn't call portable SIMD "basic" at all for instance). > Like just earlier today I would have reached for bool::toggle which not only is unstable, but is also newly added as of like a month ago! This is very likely not the kind…

> This is very likely not the kind of feature that will stay on nightly for a long time, but is instead one of the many convenience feature that land on stable every release.

Easy example of a basic method that has been unstable for a really long time: [T]::as_slice [0] since 2024 [1]. Apparently, stabilization was attempted earlier this year [2] but was then rolled back [3]. While clearly it was not yet ready for stabilization, it still took over a year before the first attempt.

Another one: Option::zip_with [4] since 2020 because nobody's figured out if it's worth having over .zip(...).map(...). Option::zip was actually stabilized [5] later in 2020 but Option::zip_with has since been sitting in limbo for over five years.

Another one: ::as_ptr also since 2020 [6]. I can't remember if there's an alternative now but dealing with slice pointers without relying on unstable methods has historically been very difficult/annoying. I ran into a bunch of this kinda stuff while working on a crate for iterating over rows/columns of image buffer subregions, because I wanted to use and support slice pointers. (Specifically I think getting the length of the slice pointer was nearly impossible without invoking UB, because constructing a reference (which was the only safe way to access a len method) could break aliasing rules. However I think the len method on slice pointers was stabilized a while ago so that particular problem is no more.) Speaking of which, ::split_at_mut has been unstable since 2022 [7]...

I'm not saying there's no reason for any of this, just that as a Rust developer it's been frustrating. There are enough of these all over the place that it feels like a real occurring problem, even if it's not reasonable to expect a volunteer open-source project to pay full attention to everything ever.

[0]: https://doc.rust-lang.org/std/primitive.slice.html#method.as...

[1]: https://github.com/rust-lang/rust/issues/130366

[2]: https://github.com/rust-lang/rust/pull/151603

[3]: https://github.com/rust-lang/rust/pull/152963

[4]: https://github.com/rust-lang/rust/issues/70086

[5]: https://github.com/rust-lang/rust/pull/72938

[6]: https://github.com/rust-lang/rust/issues/74265

[7]: https://github.com/rust-lang/rust/issues/95595

> Can't you tell it to use stable as the default target, and use nightly manually in cargo?

Are you saying it doesn't suggest unstable features when using a stable toolchain? That was not my experience before I started using nightly.

Post reply on HN