Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Async/Await on the GPU

vectorware.com

Post reply on HN
Print view View original

21–30 of 61 posts

Re: Async/Await on the GPU

#21

Post by nxobject »

zozbot234 wrote: post #3

I'm not quite seeing the real benefit of this. Is the idea that warps will now be able to do work-stealing and continuation-stealing when running heterogenous parallel workloads? But that requires keeping the async function's state in GPU-wide shared memory, which is generally a scarce resource.

God, as someone who took their elective on graphics program when GPGPU and computer shaders first became a thing, reading this makes me realize I definitely need an update on what modern GPU uarchs are like now.

Re: heterogenous workload: I'm told by a friend in HPC that the old advice about avoiding diverging branches within warps is no longer much of an issue – is that true?

Re: Async/Await on the GPU

#22

Post by zozbot234 »

Arch485 wrote: post #10

Very cool! Is the goal with this project (generally, not specifically async) to have an equivalent to e.g. CUDA, but in Rust? Or is there another intended use-case that I'm missing?

The closest equivalent to that is rust-gpu, which this project is pretty closely involved with.

Re: Async/Await on the GPU

#23

Post by zozbot234 »

nxobject wrote: post #21
zozbot234 wrote: post #3

I'm not quite seeing the real benefit of this. Is the idea that warps will now be able to do work-stealing and continuation-stealing when running heterogenous parallel workloads? But that requires keeping the async function's state in GPU-wide shared memory, which is generally a scarce resource.

God, as someone who took their elective on graphics program when GPGPU and computer shaders first became a thing, reading this makes me realize I definitely need an update on what modern GPU uarchs are like now. Re: heterogenous workload: I'm told by a friend in HPC that the old advice about avoiding diverging branches within warps is no longer much of an issue – is that true?

That advice applies within warps, to single 'threads' (effectively SIMD lanes) whereas the article is consistently about running heterogenous tasks on different warps.

Re: Async/Await on the GPU

#24

Post by Cieric »

zozbot234 wrote: post #20
Cieric wrote: post #18

Earlier quoted context omitted.

I'm not even sure a 32 wide array would be good either since on AMD warps are 64 wide. I wouldn't go fully towards auto vectorization with though.

Warp SIMD-width should be a build-time constant. You'd be using a variable-length vector-like interface that gets compiled down to a specified length as part of building the code.

Now that I could agree with, the only place where hiccups have started to occur are with wave intrinsics where you can share data between thread in a wave without halting execution. I'm not sure disallowing it would be the best idea as it cuts out possible optimizations, but outright allowing it without the user knowing the number of lanes can cause it's own problems. My job is the fun time of fixing issues in other peoples code related to all of this. I have no stakes in rust though, I'd rather write a custom spirv compiler.

Re: Async/Await on the GPU

#25

Post by zozbot234 »

Cieric wrote: post #24
zozbot234 wrote: post #20

Earlier quoted context omitted.

Warp SIMD-width should be a build-time constant. You'd be using a variable-length vector-like interface that gets compiled down to a specified length as part of building the code.

Now that I could agree with, the only place where hiccups have started to occur are with wave intrinsics where you can share data between thread in a wave without halting execution. I'm not sure disallowing it would be the best idea as it cuts out possible optimizations, but outright allowing it without the user knowing the number of lanes can cause it's own problems. My job is the fun time of fixing issues in other…

A compile time constant can still be surfaced to the user though. The code would simply be written to take the actual value into account and this would be reflected during the build.

Re: Async/Await on the GPU

#26

Post by LegNeato »

GZGavinZhao wrote: post #19

One concern I have is that this async/await approach is not "AOT"-enough like the Triton approach, in the sense that you know how to most efficiently schedule the computations on which warps since you know exactly what operations you'll be performing at compile time. Here with the async/await approach, it seems like there needs to be manual book-keeping at runtime to know what has finished, what has not, and _then_ c…

Doing things at compile time / AOT is almost always better for perf. We believe async/await and futures enables more complex programs and doing things you couldn't easily do on the GPU before. Less about performance and more about capability (though we believe async/await perf will be better in some cases, time will tell).

Re: Async/Await on the GPU

#28

Post by Cieric »

zozbot234 wrote: post #25
Cieric wrote: post #24

Earlier quoted context omitted.

Now that I could agree with, the only place where hiccups have started to occur are with wave intrinsics where you can share data between thread in a wave without halting execution. I'm not sure disallowing it would be the best idea as it cuts out possible optimizations, but outright allowing it without the user knowing the number of lanes can cause it's own problems. My job is the fun time of fixing issues in other…

A compile time constant can still be surfaced to the user though. The code would simply be written to take the actual value into account and this would be reflected during the build.

I don't have a lot of faith there, but that's mainly due to my experience being correcting peoples assumption that all gpus waves are 32 lanes. I might be biased there specifically since it's my job to fix those issues though.

Re: Async/Await on the GPU

#29

Post by zozbot234 »

GZGavinZhao wrote: post #19

One concern I have is that this async/await approach is not "AOT"-enough like the Triton approach, in the sense that you know how to most efficiently schedule the computations on which warps since you know exactly what operations you'll be performing at compile time. Here with the async/await approach, it seems like there needs to be manual book-keeping at runtime to know what has finished, what has not, and _then_ c…

If you can tell deterministically whether an 'async' computation is going to be finished, you can most likely use a type-system-like static analysis to ensure that programs are scheduled correctly and avoid any reference to values that are yet to be computed. But this is not possible in many cases, where dynamic scheduling will be preferable.

Re: Async/Await on the GPU

#30

Post by textlapse »

zozbot234 wrote: post #17
textlapse wrote: post #11

Earlier quoted context omitted.

warp is expensive - essentially it's running a 'don't run code' to maintain SIMT. GPUs are still not practically-Turing-complete in the sense that there are strict restrictions on loops/goto/IO/waiting (there are a bunch of band-aids to make it pretend it's not a functional programming model). So I am not sure retrofitting a Ferrari to cosplay an Amazon delivery van is useful other than for tech showcase? Good tech s…

I think you're conflating GPU 'threads' and 'warps'. GPU 'threads' are SIMD lanes that are all running with the exact same instructions and control flow (only with different filtering/predication), whereas GPU warps are hardware-level threads that run on a single compute unit. There's no issue with adding extra "don't run code" when using warps, unlike GPU threads.

My understanding of warp (https://docs.nvidia.com/cuda/cuda-programming-guide/01-intro...) is that you are essentially paying the cost of taking both the branches.

I understand with newer GPUs, you have clever partitioning / pipelining in such a way block A takes branch A vs block B that takes branch B with sync/barrier essentially relying on some smart 'oracle' to schedule these in a way that still fits in the SIMT model.

It still doesn't feel Turing complete to me. Is there an nvidia doc you can refer me to?

Post reply on HN