Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CUDA-oxide: Nvidia's official Rust to CUDA compiler

nvlabs.github.io

Post reply on HN
Print view View original

31–40 of 132 posts

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#31

Post by the__alchemist »

cyber_kinetist wrote: post #10

I'm quite interested in how they dealt with Rust's memory model, which might not neatly map to CUDA's semantics. Curious what the differences are compared to CUDA C++, and if the Rust's type system can actually bring more safety to CUDA (I do think writing GPU kernels is inherently unsafe, it's just too hard to create a safe language because of how the hardware works, and because of the fact that you're hyper-optimiz…

I think it depends on the objective. My pattern-matching brain says there will be interest in addressing this.

From my perspective of someone who writes applications in Rust and sometimes wants to use GPU compute in these applications: I don't care. If we can leverage the memory model or ownership model in a low-friction way, that's fine. If it makes it a high friction experience, I would prefer not to do it that way.

The baseline is IMO how Cudarc currently does it. I don't think there is much memory management involved; it's just imperative syntax wrapping FFI, and some lines in the build script to invoke nvcc if the kernels change.

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#32

Post by the__alchemist »

arpadav wrote: post #2

This is amazing.. ive been working with custom CUDA kernels and https://crates.io/crates/cudarc for a long time, and this honestly looks like it could be a near drop-in replacement. im especially curious how build times would compare? Most Rust CUDA crates obv rely on calling CMake or nvcc, which can make compilation painfully slow. coincidentally, just last week i was profiling build times and found that tools like…

Cudarc slaps!

> Most Rust CUDA crates obv rely on calling CMake or nvcc, which can make compilation painfully slow.

I anecdotally haven't hit this; see the `cuda_setup` crate I made to handle the build scripts; it is a simple `build.rs` which only recompiles if the file changes, and it's a tiny compile time (compared to the rust CPU-side code)

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#33

Post by Almondsetat »

whatever1 wrote: post #11

Why do we bother with programming languages today? Why not have the LLMs just write assembly code and skip the human readable part? We are not reviewing it anymore anyway.

Feel free to post a project of yours where you gave a bunch of prompts to an LLM and it produced a working application written in assembly without you having to check for anything

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#35

Post by wrs »

cyber_kinetist wrote: post #10

I'm quite interested in how they dealt with Rust's memory model, which might not neatly map to CUDA's semantics. Curious what the differences are compared to CUDA C++, and if the Rust's type system can actually bring more safety to CUDA (I do think writing GPU kernels is inherently unsafe, it's just too hard to create a safe language because of how the hardware works, and because of the fact that you're hyper-optimiz…

This is explained in some detail in the docs. There is a safe layer, a mostly safe layer, and an unsafe layer. Some clunkiness is needed for safe-yet-parallel work that they couldn’t easily fit into the Rust Send/Sync model.

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#36

Post by [deleted] »

jauntywundrkind wrote: post #27
arpadav wrote: post #2

This is amazing.. ive been working with custom CUDA kernels and https://crates.io/crates/cudarc for a long time, and this honestly looks like it could be a near drop-in replacement. im especially curious how build times would compare? Most Rust CUDA crates obv rely on calling CMake or nvcc, which can make compilation painfully slow. coincidentally, just last week i was profiling build times and found that tools like…

Do other people agree cuda-oxide looks like a near dorp in replacement for cudarc? That would be amazing, but probably not imo complementarily so. I am curious what distinguished cuda-oxide. Beyond it being totally under nv control.

[deleted]

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#37

Post by arpadav »

jauntywundrkind wrote: post #27
arpadav wrote: post #2

This is amazing.. ive been working with custom CUDA kernels and https://crates.io/crates/cudarc for a long time, and this honestly looks like it could be a near drop-in replacement. im especially curious how build times would compare? Most Rust CUDA crates obv rely on calling CMake or nvcc, which can make compilation painfully slow. coincidentally, just last week i was profiling build times and found that tools like…

Do other people agree cuda-oxide looks like a near dorp in replacement for cudarc? That would be amazing, but probably not imo complementarily so. I am curious what distinguished cuda-oxide. Beyond it being totally under nv control.

perhaps not drop-in, but all my workflows with cudarc have always been "i make cuda kernel, i use cudarc for ffi to said kernels, i call via rust" - which for this case is pretty analogous

briefly looking at the repo, looks like the main workflow is using rustc-codegen-cuda to convert rust -> MIR -> pliron IR -> LLVM IR -> PTX, which is embedded in the host binary, where then cuda-core loads embedded PTX at runtime onto the GPU

but, if you arent directly making cuda kernels and just want cudarc for either calling existing kernels or other cuda driver api access then cudarc is lighter-weight option? or just use one of the sub-crates in this repo like cuda-core for those apis

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#38

Post by arpadav »

the__alchemist wrote: post #32
arpadav wrote: post #2

This is amazing.. ive been working with custom CUDA kernels and https://crates.io/crates/cudarc for a long time, and this honestly looks like it could be a near drop-in replacement. im especially curious how build times would compare? Most Rust CUDA crates obv rely on calling CMake or nvcc, which can make compilation painfully slow. coincidentally, just last week i was profiling build times and found that tools like…

Cudarc slaps! > Most Rust CUDA crates obv rely on calling CMake or nvcc, which can make compilation painfully slow. I anecdotally haven't hit this; see the `cuda_setup` crate I made to handle the build scripts; it is a simple `build.rs` which only recompiles if the file changes, and it's a tiny compile time (compared to the rust CPU-side code)

i'll have to check this out, thanks!

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#39

Post by the__alchemist »

jauntywundrkind wrote: post #27
arpadav wrote: post #2

This is amazing.. ive been working with custom CUDA kernels and https://crates.io/crates/cudarc for a long time, and this honestly looks like it could be a near drop-in replacement. im especially curious how build times would compare? Most Rust CUDA crates obv rely on calling CMake or nvcc, which can make compilation painfully slow. coincidentally, just last week i was profiling build times and found that tools like…

Do other people agree cuda-oxide looks like a near dorp in replacement for cudarc? That would be amazing, but probably not imo complementarily so. I am curious what distinguished cuda-oxide. Beyond it being totally under nv control.

I am observing the same from the article... is it heavily inspired by Cudarc, i.e. is this intentional, or are we reading too much into this, given Cudarc is a light abstraction over the CUDA api?

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#40

Post by the__alchemist »

rvz wrote: post #3

This is a bit good for Rust if you want to use the language with CUDA. The problem is, it still doesn't really move the needle if you really don't like running closed source drivers and runtime binaries and care about open source. Continuing from this discussion [0], this only makes it a Rust or a CUDA problem rather than a Python, CUDA and a PyTorch one if there bug in one of them. Yet at the end of the day, it stil…

IMO this has nothing to do with open source as an ideology; just a practical (and official?) lib for adding GPU interaction to your rust programs.
Post reply on HN