Earlier quoted context omitted.
Rust GPU libraries such as wgpu and ash rely on external libraries such as vulkan-loader to load the actual ICDs, but for some reason Rust people really love dlopening them instead of linking to them normally. Then it's up to the consumer to configure their linker flags correctly so RPATH gets set correctly when needed, but because most people don't know how to use their linker, they usually end up with dumb hacks li…
Can you file a bug on rust-gpu? I'd love to look into it (I am unfamiliar with this area).
Rust running on every GPU
141–150 of 211 posts
Re: Rust running on every GPU
#142Earlier quoted context omitted.
Hears an idea, Get Nvidia, AMD, Intel and whoever else you can get into a room. Get LLVMs boys into the same room. Compile LLVMIR directly into hardware instructions fed into the GPU, get them to open up. Having to target an API is part of the problem, get them to allow you to write Rust that directly compiles into the code that will run on the GPU, not something that becomes something else, that becomes spirv that c…
Sounds sort of like the idea behind MLIR and it's GPU dialects. * https://mlir.llvm.org/docs/Dialects/NVGPU/ * https://mlir.llvm.org/docs/Dialects/AMDGPU/ * https://mlir.llvm.org/docs/Dialects/XeGPU/
Very likely something along those lines.
Effectively standardise passing operations off to a coprocessor. C++ is moving into that direction with stdexec and the linear algebra library and SIMD.
I don’t see why Rust wouldn’t also do that.
Effectively why must I write a GPU kernel to have an algorithm execute on the GPU, we’re talking about memory wrangling and linear algebra almost all of the time when dealing with GPU in any way whatsoever. I don’t see why we need a different interface and API layer for that.
OpenGL et al abstract some of the linear algebra away from you which is nice until you need to give a damn about the assumptions they made that are no longer valid. I would rather that code be in a library in the language of your choice that you can inspect and understand than hidden somewhere in a driver behind 3 layers of abstraction.
Re: Rust running on every GPU
#143Let's count abstraction layers: 1. Domain specific Rust code 2. Backend abstracting over the cust, ash and wgpu crates 3. wgpu and co. abstracting over platforms, drivers and APIs 4. Vulkan, OpenGL, DX12 and Metal abstracting over platforms and drivers 5. Drivers abstracting over vendor specific hardware (one could argue there are more layers in here) 6. Hardware That's a lot of hidden complexity, better hope one nev…
> It's also questionable how well performance relevant platform specifics survive all these layers.
Fair point, but one of Rust's strengths is the many zero-cost abstractions it provides. And the article talks about how the code complies to the GPU-specific machine code or IR. Ultimately the efficiency and optimization abilities of that compiler is going to determine how well your code runs, just like any other compilation process.
This project doesn't even add that much. In "traditional" GPU code, you're still going to have:
1. Domain specific GPU code in whatever high-level language you've chosen to work in for the target you want to support. (Or more than one, if you need it, which isn't fun.)
...
3. Compiler that compiles your GPU code into whatever machine code or IR the GPU expects.
4. Vulkan, OpenGL, DX12 and Metal...
5. Drivers...
6. Hardware...
So yes, there's an extra layer here. But I think many developers will gladly take on that trade off for the ability to target so many software and hardware combinations in one codebase/binary. And hopefully as they polish the project, debugging issues will become more straightforward.
Re: Rust running on every GPU
#144Very interesting. I wonder about the model of storing the GPU IR in binary for a real-world project; it seems like that could bloat the binary size a lot. I also wonder about the performance of just compiling for a target GPU AOT. These GPUs can be very different even if they come from the same vendor. This seems like it would compile to the lowest common denominator for each vendor, leaving performance on the table.…
The underlying projects support loading at runtime, so you could have as many AOT compiled kernel variants and load the one you want. Disk is cheap. You could even ship rustc if you really wanted for "JIT" (lol), but maybe not super crazy as mojo is llvm based anyway. There is of course the warmup / stuttering problem, but that is a separate issue and (sometimes) not an issue for compute vs graphics where it is a bigger issue. I have some thoughts on how to improve the status quo with things unique to Rust but too early to know.
One of the issues with GPUs as a platform is runtime probing of capabilities is... rudimentary to say the least. Rust has to deal with similar stuff with CPUs+SIMD FWIW. AOT vs JIT is not a new problem domain and there are no silver bullets only tradeoffs. Mojo hasn't solved anything in particular, their position in the solution space (JIT) has the same tradeoffs as anyone else doing JIT.
Re: Rust running on every GPU
#145> Though this demo doesn't do so, multiple backends could be compiled into a single binary and platform-specific code paths could then be selected at runtime. That’s kind of the goal, I’d assume: writing generic code and having it run on anything.
> writing generic code and having it run on anything. That has been already done successfully by Java applets in 1995. Wait, Java applets were dead by 2005, which leads me to assume that the goal is different.
> That has been already done successfully by Java applets in 1995.
The first video card with a programmable pixel shader was the Nvidia GeForce 3, released in 2001. How would Java applets be running on GPUs in 1995?
Besides, Java cannot even be compiled for GPUs as far as I know.
Re: Rust running on every GPU
#146Earlier quoted context omitted.
Say more?
Rust GPU libraries such as wgpu and ash rely on external libraries such as vulkan-loader to load the actual ICDs, but for some reason Rust people really love dlopening them instead of linking to them normally. Then it's up to the consumer to configure their linker flags correctly so RPATH gets set correctly when needed, but because most people don't know how to use their linker, they usually end up with dumb hacks li…
Isn't it typically done with Vulkan, because apps don't know which library you wanted to use later? On a multi-gpu system you may want to switch (for example) Intel/NVIDIA implementation at runtime rather than linking directly.
Re: Rust running on every GPU
#147I write native audio apps, where every cycle matters. I also need the full compute API instead of graphics shaders. Is the "Rust -> WebGPU -> SPIR-V -> MSL -> Metal" pipeline robust when it come to performance? To me, it seems brittle and hard to reason about all these translation stages. Ditto for "... -> Vulkan -> MoltenVk -> ...". Contrast with "Julia -> Metal", which notably bypasses MSL, and can use native optim…
> Is the "Rust -> WebGPU -> SPIR-V -> MSL -> Metal" pipeline robust when it come to performance?
It's basically the same concept as Apple's Clang optimizations, but for the GPU. SPIR-V is an IR just like the one in LLVM, which can be used for system-specific optimization. In theory, you can keep the one codebase to target any number of supported raster GPUs.
The Julia -> Metal stack is comparatively not very portable, which probably doesn't matter if you write Audio Unit plugins. But I could definitely see how the bigger cross-platform devs like u-he or Spectrasonics would value a more complex SPIR-V based pipeline.
Re: Rust running on every GPU
#148Very interesting. I wonder about the model of storing the GPU IR in binary for a real-world project; it seems like that could bloat the binary size a lot. I also wonder about the performance of just compiling for a target GPU AOT. These GPUs can be very different even if they come from the same vendor. This seems like it would compile to the lowest common denominator for each vendor, leaving performance on the table.…
The underlying projects support loading at runtime, so you could have as many AOT compiled kernel variants and load the one you want. Disk is cheap. You could even ship rustc if you really wanted for "JIT" (lol), but maybe not super crazy as mojo is llvm based anyway. There is of course the warmup / stuttering problem, but that is a separate issue and (sometimes) not an issue for compute vs graphics where it is a big…
> The underlying projects support loading at runtime, so you could have as many AOT compiled kernel variants and load the one you want.
I'm not sure I understand. What underlying projects? The only reference to loading at runtime I see on the post is loading the AOT-compiled IR.
> There is of course the warmup / stuttering problem, but that is a separate issue and (sometimes) not an issue for compute vs graphics where it is a bigger issue.
It should be worth noting that Mojo itself is not JIT compiled; Mojo has a GPU infrastructure that can JIT compile Mojo code at runtime [1].
> One of the issues with GPUs as a platform is runtime probing of capabilities is... rudimentary to say the least.
Also not an issue in Mojo when you combine the selective JIT compilation I mentioned with powerful compile-time programming [2].
1. https://docs.modular.com/mojo/manual/gpu/intro-tutorial#4-co... - here the kernel "print_threads" will be jit compiled
Re: Rust running on every GPU
#149Earlier quoted context omitted.
Sounds sort of like the idea behind MLIR and it's GPU dialects. * https://mlir.llvm.org/docs/Dialects/NVGPU/ * https://mlir.llvm.org/docs/Dialects/AMDGPU/ * https://mlir.llvm.org/docs/Dialects/XeGPU/
Very likely something along those lines. Effectively standardise passing operations off to a coprocessor. C++ is moving into that direction with stdexec and the linear algebra library and SIMD. I don’t see why Rust wouldn’t also do that. Effectively why must I write a GPU kernel to have an algorithm execute on the GPU, we’re talking about memory wrangling and linear algebra almost all of the time when dealing with GP…
>I would rather that code be in a library in the language of your choice that you can inspect and understand than hidden somewhere in a driver behind 3 layers of abstraction.
I agree that, that would be ideal. Hopefully, that can happen one day with c++, rust and other languages. So far Mojo seems to be the only language close to that vision.
Re: Rust running on every GPU
#150> Existing no_std + no alloc crates written for other purposes can generally run on the GPU without modification. Wow. That at first glance seems to unlock ALOT of interesting ideas.
I guess performance would be very different if things were initially assumed to run on a cpu