Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust running on every GPU

rust-gpu.github.io

Post reply on HN
Print view View original

171–180 of 211 posts

Re: Rust running on every GPU

#171

Post by reactordev »

5pl1n73r wrote: post #154
reactordev wrote: post #123

This is amazing. With time, we should be able to write GPU programs semantically identical to user land programs. The implications of this for inference is going to be huge.

CUDA programming consists of making a kernel parameterized by its thread id which is used to slightly alter its behavior while it executes on many hundreds of GPU cores; it's very different than general purpose programming. Memory and branching behave differently there. I'd say at best, it will be like traditional programs and libraries with multiple incompatible event loops.

I write CUDA code... I'm aware of it's execution model. In this context, having a rust program act as a kernel to receive inputs and send outputs is still awesome.

Re: Rust running on every GPU

#172

Post by johncolanduoni »

ducktective wrote: post #78
pjmlp wrote: post #52

Earlier quoted context omitted.

Khronos APIs are the C++ of graphics programming, there is a reason why professional game studios never do political wars on APIs. Decades of exerience building cross platform game engines since the days of raw assembly programming across heterogeneous computer architectures. What matters are game design and IP, that they eventually can turn into physical assets like toys, movies, collection assets. Hardware abstract…

I think you are very experienced in this subject. Can you explain what's wrong with WebGPU? Doesn't it utilize like 80% of the cool features of the modern GPUs? Games and ambitious graphics-hungry applications aside, why aren't we seeing more tech built on top of WebGPU like GUI stacks? Why aren't we seeing browsers and web apps using it? Do you recommended learning it (considering all the things worth learning nowad…

For beginner-to-intermediate raster use cases where you want to direct the GPU yourself, WebGPU will cover most of what you're looking for with reasonably low cognitive overhead. It's at a roughly DX11/Metal level of abstraction, but with some crucial modernizations that reduce friction between WebGPU and the underlying APIs (pipeline layouts being the biggest one).

The main things it's missing for more advanced raster use cases aren't really GPU features as such, but low-level control over the driver. Unlike Vulkan/DX12, it does not permit (or require) you to manage allocations of individual resources in GPU memory, or perform your own CPU/GPU and GPU/GPU synchronization. The rationale (as I recall from some of the people working on the standard talking on GitHub issues or somewhere similar) was that the computational overhead of verifying that you've done these things correctly erases the gains from giving the user this level of control. For the web platform, verification that you're not going to corrupt memory or expose undefined driver behavior is non-negotiable. This explanation makes sense to me, and I haven't really heard anyone put forward a generalizable scheme that allows you Vulkan levels of control with efficient verification.

Re: Rust running on every GPU

#174

Post by shmerl »

LegNeato wrote: post #25
vouwfietsman wrote: post #3

Certainly impressive that this is possible! However, for my use cases (running on arbitrary client hardware) I generally distrust any abstractions over the GPU api, as the entire point is to leverage the low level details of the gpu. Treating those details as a nuisance leads to bugs and performance loss, because each target is meaningfully different. To overcome this, a similar system should be brought forward by th…

Rust is a system language, so you should have the control you need. We intend to bring GPU details and APIs into the language and core / std lib, and expose GPU and driver stuff to the `cfg()` system. (Author here)

Do you get any interest from big players like AMD? I'm surprised that they didn't start such initiative, but I guess they can as well back yours.

Re: Rust running on every GPU

#175

Post by shmerl »

Ygg2 wrote: post #130
jpc0 wrote: post #126

Earlier quoted context omitted.

Hears an idea, Get Nvidia, AMD, Intel and whoever else you can get into a room. Get LLVMs boys into the same room. Compile LLVMIR directly into hardware instructions fed into the GPU, get them to open up. Having to target an API is part of the problem, get them to allow you to write Rust that directly compiles into the code that will run on the GPU, not something that becomes something else, that becomes spirv that c…

Hell will freeze over, then go into negative Kelvin temperatures before you see nVidia agreeing in earnest to do so. They make too much money on NOT GETTING COMMODITIZED. nVidia even changed CUDA to make API not compatible with interpreters. It's the same reason Safari is in such a sorry state. Why make web browser better, when it could cannibalize your app store?

Yeah, Nvidia can get lost with their CUDA moat. But AMD should be interested.

Re: Rust running on every GPU

#176

Post by shmerl »

chrisldgk wrote: post #11

Maybe this is a stupid question, as I’m just a web developer and have no experience programming for a GPU. Doesn’t WebGPU solve this entire problem by having a single API that’s compatible with every GPU backend? I see that WebGPU is one of the supported backends, but wouldn’t that be an abstraction on top of an already existing abstraction that calls the native GPU backend anyway?

This isn't about GPU APIs as far as I understand, but about having a high quality language for GPU programs. Think Rust replacing GLSL. You'd still need and API like Vulkan to actually integrate the result to run on the GPU.

Re: Rust running on every GPU

#177

Post by legends2k »

Voultapher wrote: post #34

Let's count abstraction layers: 1. Domain specific Rust code 2. Backend abstracting over the cust, ash and wgpu crates 3. wgpu and co. abstracting over platforms, drivers and APIs 4. Vulkan, OpenGL, DX12 and Metal abstracting over platforms and drivers 5. Drivers abstracting over vendor specific hardware (one could argue there are more layers in here) 6. Hardware That's a lot of hidden complexity, better hope one nev…

Even games, epitome of performance, have 5 levels of abstraction (including your 4, 5, 6 + an engine layer + game code). This isn't new in GPU/Graphics programming IMHO.

Re: Rust running on every GPU

#178

Post by ashdksnndck »

Ygg2 wrote: post #130
jpc0 wrote: post #126

Earlier quoted context omitted.

Hears an idea, Get Nvidia, AMD, Intel and whoever else you can get into a room. Get LLVMs boys into the same room. Compile LLVMIR directly into hardware instructions fed into the GPU, get them to open up. Having to target an API is part of the problem, get them to allow you to write Rust that directly compiles into the code that will run on the GPU, not something that becomes something else, that becomes spirv that c…

Hell will freeze over, then go into negative Kelvin temperatures before you see nVidia agreeing in earnest to do so. They make too much money on NOT GETTING COMMODITIZED. nVidia even changed CUDA to make API not compatible with interpreters. It's the same reason Safari is in such a sorry state. Why make web browser better, when it could cannibalize your app store?

Hmm. Maybe the opportunity would be more like AMD, Intel, and the various AI labs and big tech get together, and by their powers combined figure out a way to stop giving NVIDIA their margin?

Re: Rust running on every GPU

#179

Post by tucnak »

LegNeato wrote: post #157
tucnak wrote: post #153

Earlier quoted context omitted.

I take it you're the maintainer. Firstly, congrats on the work done, for the open source people are a small crowd, and determination of Rust teams here is commendable. On the other hand, I'm struggling to see the unique value proposition. What is your motivation with Rust-GPU? Graphics or general-purpose computing? If it's the latter, at least from my POV, I would struggle to justify going up against a daunting umbre…

Thanks! I'm personally focused on compute, while other contributors are focused on graphics. I believe GPUs are the future of computing. I think the tooling, languages, and ecosystems of GPUs are very bad compared to CPUs. Partially because they are newer, partially because they are different, and partially because for some reason the expectations are so low. So I intend to upset the status quo.

Have you considered post-GPU accelerators? For large-scale machine learning, TPU's have won, basically. There are new vendors like Tenstorrent offering completely new (and much simpler) computing hardware. GPU's may as well live on borrowed time as far as compute is concerned.

Re: Rust running on every GPU

#180

Post by pjmlp »

jpc0 wrote: post #142
bobajeff wrote: post #138

Earlier quoted context omitted.

Sounds sort of like the idea behind MLIR and it's GPU dialects. * https://mlir.llvm.org/docs/Dialects/NVGPU/ * https://mlir.llvm.org/docs/Dialects/AMDGPU/ * https://mlir.llvm.org/docs/Dialects/XeGPU/

Very likely something along those lines. Effectively standardise passing operations off to a coprocessor. C++ is moving into that direction with stdexec and the linear algebra library and SIMD. I don’t see why Rust wouldn’t also do that. Effectively why must I write a GPU kernel to have an algorithm execute on the GPU, we’re talking about memory wrangling and linear algebra almost all of the time when dealing with GP…

Guess which companies have been driving senders / receivers work.
Post reply on HN