111–120 of 132 posts
https://nvlabs.github.io/cuda-oxide/gpu-safety/the-safety-mo... > A GPU kernel runs thousands of threads that all see the same memory at the same time. On a CPU, Rust prevents data races through ownership and borrowing – one mutable reference, no aliases, enforced at compile time. On a GPU, you have 2048 threads per SM, all launched from the same function, all pointing at the same output buffer. The borrow checker wa…
> (em dash) no DSLs, no foreign language bindings, just Rust. Official CUDA port and they couldn't even bother with the introductory paragraph. Okay, I'll try to ignore it and read the docs. Hey a custom IR, this sounds interesti- > MLIR’s implementation, however, is C++ with a side of TableGen, a build system that requires you to compile all of LLVM, and debugging sessions that make you question your career choices.…
They also named it CUDA-oxide, flaunting their ignorance of what Rust lang is named after (fungi, not oxidation).
Oh lord. If this is the trend, I probably can't avoid improving my Rust language knowledge in the long term. I hate reading Rust so much right now. I guess I just have to get over that hump.
Learning Rust is more alike to learning a new programming paradigm (e.g. functional when you only know imperative) than a new language with different syntax only. If you ignore that and try to jump directly to writing code more or less the same way as you used to, it will be painful. So take it slow and follow along with The Book ( https://doc.rust-lang.org/book/ ). It all makes sense eventually and is very much wort…
Earlier quoted context omitted.
Even as someone who uses a lot of AI, if you can't be bothered to at least give it a prompt like "Go through the documentation and comments in detail and remove any obvious AI shibboleths like emdashes, it's not x it's y, rule-of-three, 'delve', excessive grandiosity and flourishes, boldness, bullet points, etc", you should receive a brisk kick in the rear.
I'd be curious to know if there is a list of these "AI shibboleths" somewhere
I'm quite interested in how they dealt with Rust's memory model, which might not neatly map to CUDA's semantics. Curious what the differences are compared to CUDA C++, and if the Rust's type system can actually bring more safety to CUDA (I do think writing GPU kernels is inherently unsafe, it's just too hard to create a safe language because of how the hardware works, and because of the fact that you're hyper-optimiz…
Whether it is a convenient language for GPU programming probably remains to be seen, but I definitely wouldn’t be surprised if you could make a decent DSL-like API for writing safe code that leverages the full spectrum of GPU oddities. That’s what CUDA is, right?
I wonder what it means for Slang[0]. Presumably the point is that people want to do GPU programming with a more modern language. But now you can just use Rust... (Disclaimer: I like Slang a lot.) [0]: https://shader-slang.org/
Stuff like descriptor sets, resource registers, dispatch limitations, …
Weird. There's a recent NVIDIA MLIR that is quite good and fast. Or they could target the even easier and more recent/fashionable tile IR [1] used by CuTile [2] (a little bit higher level but significantly easier to target, only loses on epilogue fusion and similar).
Earlier quoted context omitted.
Do other people agree cuda-oxide looks like a near dorp in replacement for cudarc? That would be amazing, but probably not imo complementarily so. I am curious what distinguished cuda-oxide. Beyond it being totally under nv control.
perhaps not drop-in, but all my workflows with cudarc have always been "i make cuda kernel, i use cudarc for ffi to said kernels, i call via rust" - which for this case is pretty analogous briefly looking at the repo, looks like the main workflow is using rustc-codegen-cuda to convert rust -> MIR -> pliron IR -> LLVM IR -> PTX, which is embedded in the host binary, where then cuda-core loads embedded PTX at runtime o…
cudarc is a host-side CUDA API for Rust: loading modules, managing contexts/streams/events/memory, launching kernels, and accessing CUDA libraries/driver APIs. If your workflow is “I already have CUDA C++/PTX/CUBIN kernels and want to call them from Rust”, cudarc is a very natural fit.
cuda-oxide is focused on the other side of the problem: writing the GPU kernel itself in Rust and compiling it through rustc/MIR into GPU code. The generated PTX is then embedded in the host binary and loaded at runtime by our host-side pieces.
We include cuda-core/cuda-host because we need an end-to-end path for “write Rust kernel, build it, launch it”, but that doesn’t mean the generated PTX is tied forever to our launcher. We’d like the PTX from cuda-oxide to be usable from other host-side CUDA APIs too, including cudarc, and we’re exploring ways to make that interop smoother.
So the short version is: cudarc is about driving CUDA from Rust; cuda-oxide is about generating CUDA device code from Rust. They’re complementary rather than replacements for each other.
We also have a short ecosystem note in the book that talks about cudarc: https://nvlabs.github.io/cuda-oxide/appendix/ecosystem.html#...
Does anyone know if this will let you share structs between host and device? That is the big thing missing so far with existing rust/CUDA workflows. (Plus the serialization/bytes barrier between them)
So the intended workflow is not “define a Rust struct on the host, define a matching CUDA C++ struct for the device, then serialize bytes between them.” It is much closer to “define `MyStruct` once in Rust, put a `DeviceBuffer` on the GPU, and write kernels that take `&[MyStruct]`, `*const MyStruct`, etc.”
There are two important pieces under the hood:
1. At the kernel boundary, cuda-oxide scalarizes aggregate parameters where needed. For example, slices become pointer + length, and simple structs can be flattened into fields for launch ABI purposes.
2. For actual struct layout, we use rustc’s computed layout rather than assuming declaration order or a C ABI. That matters because Rust is allowed to reorder/pad `repr(Rust)` structs. The device lowering carries those offsets/padding through so field access on the GPU matches the host-side layout.
So for plain data structs, nested structs, numeric fields, arrays, etc., yes, this is very much the goal: share the type directly instead of maintaining a separate CUDA representation or crossing a bytes/serialization boundary.
The caveat is the usual one: this does not make arbitrary host-owned Rust heap graphs GPU-addressable. A `Vec`, `String`, `Box`, trait object, or host pointer still contains an address, and that address has to refer to memory the GPU can actually access. For those cases you still need device allocation, unified/HMM memory, or a GPU-friendly representation.
But for the common “I have a Rust data type and want kernels to consume/update arrays of it” case: yes, that is exactly the kind of friction cuda-oxide is meant to remove.
This is a bit good for Rust if you want to use the language with CUDA. The problem is, it still doesn't really move the needle if you really don't like running closed source drivers and runtime binaries and care about open source. Continuing from this discussion [0], this only makes it a Rust or a CUDA problem rather than a Python, CUDA and a PyTorch one if there bug in one of them. Yet at the end of the day, it stil…
The device path is roughly: rustc frontend/MIR -> cuda-oxide’s Pliron-based IR/lowering pipeline -> LLVM IR -> PTX via LLVM’s NVPTX backend. Host code still goes through normal rustc/LLVM codegen.
You still need the NVIDIA driver/toolkit pieces to run CUDA code and load PTX on NVIDIA GPUs, so this does not change the broader CUDA ecosystem/dependency story. But it is not “Rust source handed to nvcc”; it is a Rust compiler backend generating device code.