Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LibreCUDA – Launch CUDA code on Nvidia GPUs without the proprietary runtime

github.com

Post reply on HN
Print view View original

151–160 of 162 posts

Re: LibreCUDA – Launch CUDA code on Nvidia GPUs without the proprietary runtime

#151

Post by einpoklum »

queuebert wrote: post #6
snihalani wrote: post #3

For a non cuda n00b, what problem does this solve?

Two obvious problems that come to mind are 1. Replacing the extremely bloated official packages with lightweight distribution that provides only the common functionality. 2. Paving the way for GPU support on *BSD.

It doesn't solve problem (1.) ; even when complete, this will replace the CUDA driver and its associated library - which is a very small part of CUDA. As for (2.) - this is just CUDA, not GPU use in general. I wonder whether nouveau is relevant for BSDs (I have no idea...)

Re: LibreCUDA – Launch CUDA code on Nvidia GPUs without the proprietary runtime

#152

Post by segmondy »

oaththrowaway wrote: post #146
segmondy wrote: post #72

Earlier quoted context omitted.

Some of us are running llama 405B locally already. All my GPUs are ancient Nvidai GPUs. IMO, the point of an open cuda is to force Nvidia to stop squeezing us. You get more performance for the buck for AMD. If I could run cuda on AMD, I would have bought new AMD gpus instead. Have enough people do that and Nvidia might take note and stop squeezing us for cash.

What are you using P100s or something?

many 3090s

Re: LibreCUDA – Launch CUDA code on Nvidia GPUs without the proprietary runtime

#158

Post by mike64_t »

paulmd wrote: post #86
londons_explore wrote: post #50

Earlier quoted context omitted.

The NVidia software stack has the "no use in datacenters" clause. Is this a workaround for that?

use the open kernel driver, which is MIT/GPL and thus cannot impose usage restrictions. it's worth noting that "NVIDIA software stack" is an imprecise term. the driver is the part that has the datacenter usage term, and the open-kernel-driver bypasses that. the CUDA stack itself does not have the datacenter driver clause, the only caveat is that you can't run it on third-party hardware. So ZLUDA/GpuOcelot is still ve…

To what degree the GeForce software license is applicable even if you don't download the GeForce driver is debatable. It technically governs all software downloadable from nvidia.com including CUDA. The fact that you are only presented with the CUDA license while downloading might not stop Nvidia's lawyers. You might be safe as a consumer against EULAs without the ability to review and reject, but as a company, this might not slide.

Re: LibreCUDA – Launch CUDA code on Nvidia GPUs without the proprietary runtime

#159

Post by mike64_t »

KeplerBoy wrote: post #7

What's a CUDA elf file? Is it binary SASS code, so one would still need a open source ptxas alternative?

Yes, the Nvidia SASS ISAs are not documented and emitting them is non trivial due to Nvidia GPUs not handling pipeline hazards in Hardware and requires the compiler to correctly schedule instructions to avoid race conditions. The only available code that does this can be found in MESA, but even they say "//this is bs and we know it" in a comment above their instruction latencies, which you also can't easily figure out.

Replacing ptxas is highly non trivial. I will attempt to do so, but it increasingly looks like ptxas is here to stay. I started working on a nvcc + cuda SDK replacement which already works surprisingly well for a day of work.

However, ptxas is in my sight. But I know this is something that to my knowledge nobody that wasn't fed Nvidia documentation under license has ever successfully accomplished.

Re: LibreCUDA – Launch CUDA code on Nvidia GPUs without the proprietary runtime

#160

Post by mike64_t »

georgehotz wrote: post #94

Cool to see one of these in C, particularly if it can be binary compatible. Why not s/libreCuInit/cuInit? If you are interested in open source runtimes, tinygrad has them in Python for both AMD and NVIDIA, speaking directly to the kernel through ioctls and poking the command queues. https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru... https://github.com/tinygrad/tinygrad/blob/master/tinygrad/ru...

Binary compatibility is possible, but not my main concern just yet. The CUDA api is missing length parameters left and right, often for highly problematic things such as "how long is this ELF file" and "how many parameters does this kernel need". I will definitely write wrapper headers at some point, but I don't want those hacks in the actual source code...
Post reply on HN