Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust SIMD on the GPU

vectorware.com

Post reply on HN
Print view View original

111–120 of 121 posts

Re: Rust SIMD on the GPU

#111

Post by pezezin »

MindSpunk wrote: post #87
mathisfun123 wrote: post #85

Earlier quoted context omitted.

> GPUs aren't really SIMD, they're SIMT (single instruction multiple thread) False. If they were threads they'd have their own PC. They do not - only the warp has a PC. > You just write plain scalar code and the hardware model dispatches it efficiently to SIMD execution units. Absolutely not. If you don't write coalesced loads, bank-conflict free, predication-free, cooperative code you will get worse than CPU perform…

Yes, of course writing naive code assuming each lane in a thread group is a real thread is going to cause problems, but I didn't feel like I needed to go into that level of detail replying to someone just learning about GPU internals. I tried to cover this loosely by mentioning how you need to know how it works for maximum performance. If you want to get more pedantic you also need to look at your target hardware and…

> While each lane isn't truly a thread because it doesn't have its own PC the programming model definitely tries to make it seem that way. The threads can terminate at different points too. And again, the ISA isn't a vector ISA. Your register values are scalar.

This is not correct. If you check AMD's documentation there are explicit mentions of vector registers (VGPR), vector ALUs, and vector instructions. The introduction to Chapter 2 describes it as a vector ISA.

> RDNA4 shader programs (kernels) are programs executed by the shader processor. Conceptually, the shader program is executed independently on every work-item, but in reality the processor groups up to 32 or 64 work-items into a wave, that executes the shader program on all 32 or 64 work-items in one pass ("wave32" or "wave64").

Sources:

https://gpuopen.com/amd-gpu-architecture-programming-documen...

https://docs.amd.com/v/u/en-US/rdna4-instruction-set-archite...

Re: Rust SIMD on the GPU

#112

Post by saidnooneever »

LoganDark wrote: post #78
jonkoops wrote: post #45

Earlier quoted context omitted.

Pretty common for Rust to cook things in nightly for a very long time; I wouldn't consider it a bad thing, tbh.

It's been annoying to me as an end user that so many basic things require nightly. I use nightly as my main toolchain, but enabling unstable features makes a project nightly-only, which is undesired for crates that don't already revolve around the unstable feature. I most often encounter unstable features when I reach for a basic common-sense utility method and discover that it's not stable. Like just earlier today I…

bool::toggle??

Re: Rust SIMD on the GPU

#113

Post by kbolino »

krapht wrote: post #65
IshKebab wrote: post #61

Earlier quoted context omitted.

Sure but there's no real way to use that in a portable way, at least not a way that maximises performance on every CPU you run it on. That's pretty much impossible at the moment.

Which is why I've never quite understood the appeal of portable SIMD libraries for performance-critical code. If I'm explicitly writing SIMD rather than relying on the auto-vectorizer, it's usually because I want access to the particular capabilities of the target ISA. For many problems, choosing the right instruction or instruction sequence makes a large difference. Portable SIMD abstractions necessarily expose some…

Go doesn't have auto-vectorization in the first place, so its portable simd library is at least partly there to fill the gap.

Re: Rust SIMD on the GPU

#115

Post by corysama »

dataflow wrote: post #84
corysama wrote: post #72

Earlier quoted context omitted.

When you dig through the CUDA developer docs instead of the promotional materials, you can develop a view of Nvidia GPUs as having 8-128 processing cores, each with 4 hyperthreads, running 32-lane SIMD for almost everything. Where a lane is 32 bits wide. The promotional material likes to label the individual lanes as “cores” because it sounds more impressive. And, it’s not entirely incorrect. Even the dev docs use th…

This is a fantastic explanation, thanks for writing it. It also makes me wonder something: where exactly is the biggest difference between a 32-core x86 CPU (AVX512 basically being 16 32-bit lanes) and (say) an NVIDIA GPU with ~8-16 processing cores? Like why can't the CPU compete against a GPU like that for GPU-y tasks - or can it?

On a CPU, hyperthreads are mostly replicated register banks. This allows the CPU to hold the context for 2 threads simultaneously. And, lets parts of a CPU make progress on one thread while the other thread is stalled. CPUs also has a kinda large microcode register bank that helps work around dependencies in asm instructions that reuse named registers.

On the GPU however, the hyperthreads are just a round-robin execution queue to take advantage of instruction pipelining. The register bank of a single GPU core is huge and can be flexibly divided across a variable number of thread contexts when a kernel is launched. Many thread contexts can be held in registers simultaneously in a single GPU core. That makes stalling on memory latency much less of a problem. The hardware can focus on delivering raw bandwidth with high latency and get great overall performance. This throughput-instead-of-latency trade-off extends to many other aspects of GPU design.

Re: Rust SIMD on the GPU

#116

Post by simonask »

MindSpunk wrote: post #74
6r17 wrote: post #15

My heard hurts - i was stupid enough to think that SIMD was a CPU only thing - I don't understand why it would be ported to GPU - huge kudos to managing to surprise me

It's not really obvious unless you go in depth of the details on modern GPU architecture. GPUs aren't really SIMD, they're SIMT (single instruction multiple thread). The silicon looks a lot like SIMD, but the programming model is different. If you go look at AMD's ISA docs (they're public) you'll see you don't have the equivalent of a __mm256 register like on x86. Each 'thread' just deals with single scalar values li…

> GPUs aren't really SIMD, they're SIMT (single instruction multiple thread).

This might be more confusing than it needs to be. SIMD and SIMT are not mutually exclusive.

People commonly think of things like vector registers when they talk about SIMD, and each "thread" in a GPU warp definitely deals with local vector registers. Granted, they may be slices of superwide registers shared by the whole warp, or whatever else, but from the programmer's perspective, that's a valid way to think about it.

Put another way, it would be a mistake to think that each lane of a vec4 in a shader gets processed by a separate unit.

Re: Rust SIMD on the GPU

#117

Post by simonask »

jandrewrogers wrote: post #75
camel-cdr wrote: post #37

I love how ever example of portable SIMD isn't portable. They specifies a constant SIMD width so it's non-portable. Well, not performance portable, but why are we using SIMD again?

The capabilities of various SIMD ISAs don't have enough intersection to be portable outside of relatively trivial cases. Many of the somewhat unique capabilities are load-bearing, so you want to use them on architectures that support them. Taken in whole, someone who cares about performance would be using different data structures and algorithms depending on the specific SIMD architecture and that is nearly impossibl…

> The capabilities of various SIMD ISAs don't have enough intersection to be portable outside of relatively trivial cases.

I would argue that the "trivial" cases (those relating to linear algebra in 3 dimensions) are also 95% of what people want SIMD for.

If the API can achieve cross-platform and performant vector arithmetic, dot product, and matrix multiplication in the normal ways, that already covers a lot of what people actually need.

Re: Rust SIMD on the GPU

#118

Post by nextaccountic »

jonkoops wrote: post #45
O3marchnative wrote: post #32

The author mentions Rust's portable SIMD library [0]. The only issue with portable SIMD is it's only available on nightly. I used it in my FFT crate, but we had to switch to the fearless_simd crate in order to get a portable SIMD solution that works on stable [1]. [0] https://doc.rust-lang.org/std/simd/index.html [1] https://github.com/linebender/fearless_simd

Pretty common for Rust to cook things in nightly for a very long time; I wouldn't consider it a bad thing, tbh.

It's a bad thing when there are no breaking changes done and they stabilize the exact same thing a few years later

Re: Rust SIMD on the GPU

#119

Post by zbentley »

nextaccountic wrote: post #118
jonkoops wrote: post #45

Earlier quoted context omitted.

Pretty common for Rust to cook things in nightly for a very long time; I wouldn't consider it a bad thing, tbh.

It's a bad thing when there are no breaking changes done and they stabilize the exact same thing a few years later

Eh, it's hard to prove absence, and time is a beneficial quantity here. The longer something sits on nightly, the greater the chance that bugs are identified before it reaches stable and its usage significantly increases.

A lot of bugs with SIMD libraries are in the domain of interactions, not functionality--e.g. SIMD malfunctions on rare chips, chips with previously-unseen combinations of hardware/userspace firmware/microcode behavior, compilers run in weird harnesses that lie about hardware capabilities, and so on. I assume that's the case with Rust's portable SIMD as well.

If your QA is unpredictable individual use-cases (as with most OSS projects), then there's no way to measure "testing complete" or "coverage"; letting it bake for awhile is the best approach available.

Re: Rust SIMD on the GPU

#120

Post by jodrellblank »

saidnooneever wrote: post #112
LoganDark wrote: post #78

Earlier quoted context omitted.

It's been annoying to me as an end user that so many basic things require nightly. I use nightly as my main toolchain, but enabling unstable features makes a project nightly-only, which is undesired for crates that don't already revolve around the unstable feature. I most often encounter unstable features when I reach for a basic common-sense utility method and discover that it's not stable. Like just earlier today I…

bool::toggle??

https://github.com/rust-lang/libs-team/issues/820 ?
Post reply on HN