Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust SIMD on the GPU

vectorware.com

Post reply on HN
Print view View original

121–126 of 126 posts

Re: Rust SIMD on the GPU

#121

Post by MindSpunk »

pezezin wrote: post #111
MindSpunk wrote: post #87

Earlier quoted context omitted.

Yes, of course writing naive code assuming each lane in a thread group is a real thread is going to cause problems, but I didn't feel like I needed to go into that level of detail replying to someone just learning about GPU internals. I tried to cover this loosely by mentioning how you need to know how it works for maximum performance. If you want to get more pedantic you also need to look at your target hardware and…

> While each lane isn't truly a thread because it doesn't have its own PC the programming model definitely tries to make it seem that way. The threads can terminate at different points too. And again, the ISA isn't a vector ISA. Your register values are scalar. This is not correct. If you check AMD's documentation there are explicit mentions of vector registers (VGPR), vector ALUs, and vector instructions. The introd…

A VGPR is not the same thing as a vector register like in SSE4 or AVX. Each addressed register contains a single 32-bit value. A VGPR differs from an SGPR in that each thread in a thread group can have a different value in that register. An SGPR will have a uniform value shared with all threads in a group.

An add instruction on an AMD GPU adds two scalar values. If they're in a VGPR then each thread will add two values unique to that thread. A SIMD ISA as is common on a CPU is different because an add instruction explicitly adds a vector of values. xmm1 stores 128-bits of data. VGPR[1] stores 32-bits of data vectored over 32-64 threads in a thread group.

Without special instructions a thread can't access the VGPR values stored in other threads.

Re: Rust SIMD on the GPU

#122

Post by [deleted] »

amdivia wrote: post #101
mathisfun123 wrote: post #85

Earlier quoted context omitted.

> GPUs aren't really SIMD, they're SIMT (single instruction multiple thread) False. If they were threads they'd have their own PC. They do not - only the warp has a PC. > You just write plain scalar code and the hardware model dispatches it efficiently to SIMD execution units. Absolutely not. If you don't write coalesced loads, bank-conflict free, predication-free, cooperative code you will get worse than CPU perform…

Actually not so false anymore. (But still they don't expect you to use this knowledge while coding, and you should treat all threads in a warp as moving in lockstep) > In GPUs of compute capability 7.0 and later, independent thread scheduling allows full concurrency between threads, regardless of warp. With independent thread scheduling, the GPU maintains execution state per thread, including a program counter and ca…

[deleted]

Re: Rust SIMD on the GPU

#123

Post by jandrewrogers »

simonask wrote: post #117
jandrewrogers wrote: post #75

Earlier quoted context omitted.

The capabilities of various SIMD ISAs don't have enough intersection to be portable outside of relatively trivial cases. Many of the somewhat unique capabilities are load-bearing, so you want to use them on architectures that support them. Taken in whole, someone who cares about performance would be using different data structures and algorithms depending on the specific SIMD architecture and that is nearly impossibl…

> The capabilities of various SIMD ISAs don't have enough intersection to be portable outside of relatively trivial cases. I would argue that the "trivial" cases (those relating to linear algebra in 3 dimensions) are also 95% of what people want SIMD for. If the API can achieve cross-platform and performant vector arithmetic, dot product, and matrix multiplication in the normal ways, that already covers a lot of what…

Most use cases for SIMD are non-arithmetic in nature and don't assume tidy arrays of homogeneous types. I also use it for some computational geometry but that is the least interesting use case.

SIMD is widely used throughout data infrastructure e.g. parsing data, complex constraint processing, parallel manipulation of heterogeneous data types, compression, etc. I even have an I/O scheduler written in AVX-512 that is many times faster than the scalar equivalent. The ability of SIMD to do complex manipulation of ordinary data structures several times faster than scalar code is under-rated.

While linear algebra is the current thing, database engines have been using SIMD heavily for over a decade and arguably represent the frontier. It is for these use cases that SIMD is non-portable and data infrastructure isn't going away.

Re: Rust SIMD on the GPU

#124

Post by [deleted] »

zbentley wrote: post #119
nextaccountic wrote: post #118

Earlier quoted context omitted.

It's a bad thing when there are no breaking changes done and they stabilize the exact same thing a few years later

Eh, it's hard to prove absence, and time is a beneficial quantity here. The longer something sits on nightly, the greater the chance that bugs are identified before it reaches stable and its usage significantly increases. A lot of bugs with SIMD libraries are in the domain of interactions, not functionality--e.g. SIMD malfunctions on rare chips, chips with previously-unseen combinations of hardware/userspace firmware…

[deleted]

Re: Rust SIMD on the GPU

#125

Post by jmalicki »

jonkoops wrote: post #45
O3marchnative wrote: post #32

The author mentions Rust's portable SIMD library [0]. The only issue with portable SIMD is it's only available on nightly. I used it in my FFT crate, but we had to switch to the fearless_simd crate in order to get a portable SIMD solution that works on stable [1]. [0] https://doc.rust-lang.org/std/simd/index.html [1] https://github.com/linebender/fearless_simd

Pretty common for Rust to cook things in nightly for a very long time; I wouldn't consider it a bad thing, tbh.

For these sort of things, I wish they would have some semi-stable beta or prerelease tracks other than just nightly so you could use the new features on a somewhat stable branch. If something is in nightly that is too crazy for a lot of projects to really try and exercise it because so much is changing constantly. Like a monthly or quarterly stabilization would be amazing, that may still have experimental features not making it into stable, but has a period of bugfixing w/o intentionally breaking changes to settle down.
Post reply on HN