Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CUDA-oxide: Nvidia's official Rust to CUDA compiler

nvlabs.github.io

Post reply on HN
Print view View original

101–110 of 132 posts

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#101

Post by xkevio »

magnio wrote: post #25
erk__ wrote: post #19

Earlier quoted context omitted.

There is actually work on adding autodiff to Rust, maybe not really first class citizen, but at least build in: https://doc.rust-lang.org/std/autodiff/index.html (it is still at a pre-RFC stage so it is not something that soon will be added)

Incredible, I have never heard of std::autodiff before. Isn't it rare for a programming language to provide AD within the standard library? Even Julia doesn't have it built-in, I wouldn't expect Rust out of all languages to experiment it in std.

It makes use of https://github.com/EnzymeAD/enzyme which is an LLVM plugin and since Rust also uses LLVM in its backend, we can enable this plugin in our Rust toolchain when autodiff is enabled. So, it is a bit of compiler black magic rather than a direct implementation in the standard library.

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#102

Post by zamalek »

pjmlp wrote: post #78
zamalek wrote: post #58

Earlier quoted context omitted.

My sentiment matches your exactly. I'm sick and tired of CUDA - but it's really not going to change. Could maybe be forked with some dynamic smarts, HIP is basically 1:1 with CUDA: https://github.com/amd/amd-lab-notes/blob/release/hipify%2Fs...

Does it support a graphical GPU debugging for C++, Fortran and Python JIT GPU code? Otherwise it isn't 1:1 with CUDA, and I am not counting everything else on CUDA ecosystem

Debugging (gdb, but GUI frontends exist for gdb in general): https://rocm.docs.amd.com/projects/HIP/en/latest/how-to/debu...

Fortran: https://github.com/ROCm/hipfort

Python (not sure about JIT): https://rocm.docs.amd.com/projects/hip-python/en/latest/

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#103

Post by hellohello2 »

hellohello2 wrote: post #17
whatever1 wrote: post #11

Why do we bother with programming languages today? Why not have the LLMs just write assembly code and skip the human readable part? We are not reviewing it anymore anyway.

I get what you mean but I think if anything AI pairs extremely well with strongly typed languages that are at times cumbersome for humans, but decrease the latency at which AI can get feedback on its code. In my (very) limited experience Rust is an excellent target for AI codegen.

(I meant statically typed / high level of type safety here not strongly typed)

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#104

Post by jaggederest »

jameslk wrote: post #100
jaggederest wrote: post #95

Earlier quoted context omitted.

Even as someone who uses a lot of AI, if you can't be bothered to at least give it a prompt like "Go through the documentation and comments in detail and remove any obvious AI shibboleths like emdashes, it's not x it's y, rule-of-three, 'delve', excessive grandiosity and flourishes, boldness, bullet points, etc", you should receive a brisk kick in the rear.

I'd be curious to know if there is a list of these "AI shibboleths" somewhere

Not really but they jump right out at you after a few minutes chatting with it. I also asked the AI and it was pretty subjectively accurate, especially if you force it to cross reference with web searches and especially google's ngram corpus (you can readily see that 'delve' and some of the other rhetorical constructs are quite uncommon in human speech)

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#105

Post by mswphd »

magnio wrote: post #25
erk__ wrote: post #19

Earlier quoted context omitted.

There is actually work on adding autodiff to Rust, maybe not really first class citizen, but at least build in: https://doc.rust-lang.org/std/autodiff/index.html (it is still at a pre-RFC stage so it is not something that soon will be added)

Incredible, I have never heard of std::autodiff before. Isn't it rare for a programming language to provide AD within the standard library? Even Julia doesn't have it built-in, I wouldn't expect Rust out of all languages to experiment it in std.

You can read some motivation for it at the following link

https://rust-lang.github.io/rust-project-goals/2024h2/Rust-f...

note that it also discusses `std::offload`, which might also be of interest.

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#106

Post by argee »

debugnik wrote: post #70

> (em dash) no DSLs, no foreign language bindings, just Rust. Official CUDA port and they couldn't even bother with the introductory paragraph. Okay, I'll try to ignore it and read the docs. Hey a custom IR, this sounds interesti- > MLIR’s implementation, however, is C++ with a side of TableGen, a build system that requires you to compile all of LLVM, and debugging sessions that make you question your career choices.…

They also named it CUDA-oxide, flaunting their ignorance of what Rust lang is named after (fungi, not oxidation).

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#107

Post by debugnik »

argee wrote: post #106
debugnik wrote: post #70

> (em dash) no DSLs, no foreign language bindings, just Rust. Official CUDA port and they couldn't even bother with the introductory paragraph. Okay, I'll try to ignore it and read the docs. Hey a custom IR, this sounds interesti- > MLIR’s implementation, however, is C++ with a side of TableGen, a build system that requires you to compile all of LLVM, and debugging sessions that make you question your career choices.…

They also named it CUDA-oxide, flaunting their ignorance of what Rust lang is named after (fungi, not oxidation).

That's a lost battle even in the Rust community: Firefox's oxidation, Ferrous Systems, Redox, OxidOS, OxCaml (OCaml extensions partly inspired by Rust)… and every crate referencing oxidation in its name.

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#108

Post by pjmlp »

zamalek wrote: post #102
pjmlp wrote: post #78

Earlier quoted context omitted.

Does it support a graphical GPU debugging for C++, Fortran and Python JIT GPU code? Otherwise it isn't 1:1 with CUDA, and I am not counting everything else on CUDA ecosystem

Debugging (gdb, but GUI frontends exist for gdb in general): https://rocm.docs.amd.com/projects/HIP/en/latest/how-to/debu... Fortran: https://github.com/ROCm/hipfort Python (not sure about JIT): https://rocm.docs.amd.com/projects/hip-python/en/latest/

All those are far from the 1:1 CUDA experience.

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#109

Post by rudedogg »

strbean wrote: post #18
whatever1 wrote: post #11

Why do we bother with programming languages today? Why not have the LLMs just write assembly code and skip the human readable part? We are not reviewing it anymore anyway.

A lot of really good reasons: 1) Higher level code is easier for LLMs to review and iterate upon. The more the intent is clear from the code, the easier it is for humans and LLMs to work with. 2) LLMs get stuck or fail to solve a problem sometimes. It is preferable to have artifacts that humans can grok without the massive extra effort of parsing out assembly code. 3) Assembly code varies massively across targets. We…

> 1) Higher level code is easier for LLMs to review and iterate upon. The more the intent is clear from the code, the easier it is for humans and LLMs to work with.

The counter-argument, and one that matches my experience is working at a lower level is actually beneficial for LLMs since they can see the whole picture and don’t have to guess at abstractions.

Re: CUDA-oxide: Nvidia's official Rust to CUDA compiler

#110

Post by bigyabai »

vlovich123 wrote: post #45
bigyabai wrote: post #24

Earlier quoted context omitted.

In AI-focused fields like business analytics and data science, yeah.

The claim is that people are running CUDA on Windows for business analytics and data science? This feels less likely an accurate picture and more likely any mass data processing is already happening on Linux K8s clusters.

The K8s clusters do exist, but I've never met anyone in my life that develops their Jupyter notebooks from their business' Kubernetes setup. Most of them don't even use WSL, to my chagrin (and to their detriment).
Post reply on HN