Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rust running on every GPU

rust-gpu.github.io

Post reply on HN
Print view View original

191–200 of 211 posts

Re: Rust running on every GPU

#191

Post by max-privatevoid »

viraptor wrote: post #146
max-privatevoid wrote: post #110

Earlier quoted context omitted.

Rust GPU libraries such as wgpu and ash rely on external libraries such as vulkan-loader to load the actual ICDs, but for some reason Rust people really love dlopening them instead of linking to them normally. Then it's up to the consumer to configure their linker flags correctly so RPATH gets set correctly when needed, but because most people don't know how to use their linker, they usually end up with dumb hacks li…

Isn't it typically done with Vulkan, because apps don't know which library you wanted to use later? On a multi-gpu system you may want to switch (for example) Intel/NVIDIA implementation at runtime rather than linking directly.

That's the point of ICDs. That's why you link to e.g. vulkan-loader instead of directly to the driver library.

Re: Rust running on every GPU

#192

Post by vouwfietsman »

LegNeato wrote: post #25
vouwfietsman wrote: post #3

Certainly impressive that this is possible! However, for my use cases (running on arbitrary client hardware) I generally distrust any abstractions over the GPU api, as the entire point is to leverage the low level details of the gpu. Treating those details as a nuisance leads to bugs and performance loss, because each target is meaningfully different. To overcome this, a similar system should be brought forward by th…

Rust is a system language, so you should have the control you need. We intend to bring GPU details and APIs into the language and core / std lib, and expose GPU and driver stuff to the `cfg()` system. (Author here)

> Rust is a system language, so you should have the control you need

I don't think this argument is promising. Its not about the power of the language, but the power of the abstractions you provide over the various GPU APIs.

In fact, I could argue one of the main selling points of Rust (memory safety) has limited applicability in GPU land, because lifetimes are not a thing like they are in CPU land.

I'm sure there's other benefits here, not least the tooling, but certainly the language is not the main selling point...

Re: Rust running on every GPU

#193

Post by saagarjha »

LowLevelMahn wrote: post #20
littlestymaar wrote: post #19

Earlier quoted context omitted.

The performance purist don't use Cuda either though (that's why Deepseek used PTX directly). Everything is an abstraction and choosing the right level of abstraction for your usecase is a tradeoff between your engineering capacities and your performance needs.

this Rust demo also uses PTX directly During the build, build.rs uses rustc_codegen_nvvm to compile the GPU kernel to PTX. The resulting PTX is embedded into the CPU binary as static data. The host code is compiled normally.

That’s not really the same thing; it compiles through PTX rather than using inline assembly.

Re: Rust running on every GPU

#195

Post by tucnak »

LegNeato wrote: post #190
tucnak wrote: post #179

Earlier quoted context omitted.

Have you considered post-GPU accelerators? For large-scale machine learning, TPU's have won, basically. There are new vendors like Tenstorrent offering completely new (and much simpler) computing hardware. GPU's may as well live on borrowed time as far as compute is concerned.

Yes, see the proposed WG link I posted above. When I say GPU I'm using it as shorthand...indeed, I think the "graphics" part is on borrowed time and will just become fully software. It is already happening.

Tenstorrent is often criticised for having lots of abstraction layers, compilers, IR's in the middle—it's all in C++, of course. GPU's are okay, but none of them got network-on-chip capability. Some promising papers have been coming out, like SystolicAttention, etc. There's just so much stuff for GPU's, but not that much for sysolic NoC systems (TPU, TT, NPU's) I think Rust could really make an impact here. Abandon all GPU deadweight, stick to simple abstractions, assume 3d twisted torus for topology and that's it. Food for thought!

Re: Rust running on every GPU

#196

Post by tucnak »

Ygg2 wrote: post #131
tucnak wrote: post #51

Earlier quoted context omitted.

I must agree that for numerical computation (and downstream optimisation thereof) Julia is much better suited than ostensibly "systems" language such as Rust. Moreover, the compatibility matrix[1] for Rust-CUDA tells a story: there's seemingly very little demand for CUDA programming in Rust, and most parts that people love about CUDA are notably missing. If there was demand, surely it would get more traction, alas, i…

It's not just that. See CUDA EULA at https://docs.nvidia.com/cuda/eula/index.html Section 1.2 Limitations: You may not reverse engineer, decompile or disassemble any portion of the output generated using SDK elements for the purpose of translating such output artifacts to **target a non-NVIDIA platform**. Emphasis mine.

Well, believe it or not, CUDA fans only ever target NVIDIA systems... that's the whole point. However, EULA itself is completely besides the point.

Re: Rust running on every GPU

#197

Post by Archit3ch »

bigyabai wrote: post #147
Archit3ch wrote: post #44

I write native audio apps, where every cycle matters. I also need the full compute API instead of graphics shaders. Is the "Rust -> WebGPU -> SPIR-V -> MSL -> Metal" pipeline robust when it come to performance? To me, it seems brittle and hard to reason about all these translation stages. Ditto for "... -> Vulkan -> MoltenVk -> ...". Contrast with "Julia -> Metal", which notably bypasses MSL, and can use native optim…

> Is the "Rust -> WebGPU -> SPIR-V -> MSL -> Metal" pipeline robust when it come to performance? It's basically the same concept as Apple's Clang optimizations, but for the GPU. SPIR-V is an IR just like the one in LLVM, which can be used for system-specific optimization. In theory, you can keep the one codebase to target any number of supported raster GPUs. The Julia -> Metal stack is comparatively not very portable…

> The Julia -> Metal stack is comparatively not very portable

You can do "Julia -> KernelAbstractions.jl -> Metal", "Julia -> KernelAbstractions.jl -> CUDA", etc. if you need portability. This is already used by some of the numerical libraries in the ecosystem.

Re: Rust running on every GPU

#198

Post by Ygg2 »

pjmlp wrote: post #181
Ygg2 wrote: post #130

Earlier quoted context omitted.

Hell will freeze over, then go into negative Kelvin temperatures before you see nVidia agreeing in earnest to do so. They make too much money on NOT GETTING COMMODITIZED. nVidia even changed CUDA to make API not compatible with interpreters. It's the same reason Safari is in such a sorry state. Why make web browser better, when it could cannibalize your app store?

By making Web browser "better" do you mean more ChromeOS like? CUDA is great for Python as well. Maybe Intel and AMD should actually produce something worthwhile using.

> By making Web browser "better" do you mean more ChromeOS like?

Whichever part makes Safari completely fail at properly rendering Jira. A task even Firefox can do.

Re: Rust running on every GPU

#199

Post by hyperbolablabla »

vouwfietsman wrote: post #3

Certainly impressive that this is possible! However, for my use cases (running on arbitrary client hardware) I generally distrust any abstractions over the GPU api, as the entire point is to leverage the low level details of the gpu. Treating those details as a nuisance leads to bugs and performance loss, because each target is meaningfully different. To overcome this, a similar system should be brought forward by th…

What we really need is a consistent GPU ISA. If it wasn't for the fairly recent proliferation of ARM CPUs, we more or less would've rallied around x86 as the de facto ISA for general purpose compute. I'm not sure why we couldn't do the same for GPUs as well.

Re: Rust running on every GPU

#200

Post by jmaker »

willglynn wrote: post #152
slashdev wrote: post #76

This is a little crude still, but the fact that this is even possible is mind blowing. This has the potential, if progress continues, to break the vendor-locked nightmare that is GPU software and open up the space to real competition between hardware vendors. Imagine a world where machine learning models are written in Rust and can run on both Nvidia and AMD. To get max performance you likely have to break the abstra…

You might be interested in https://burn.dev , a Rust machine learning framework. It has CUDA and ROCm backends among others.

I’ve just tried the MNIST demo on that page, but at least in a third of cases it’s plainly misclassified with zero probability.

Is it due to WASM? Browser limitations? Does it impose any constraints on inference?

Post reply on HN