Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia adds native Python support to CUDA

thenewstack.io

Post reply on HN
Print view View original

121–130 of 196 posts

Re: Nvidia adds native Python support to CUDA

#121

Post by int_19h »

taminka wrote: post #31
the__alchemist wrote: post #8

Rust support next? RN I am manually [de]serializing my data structures as byte arrays to/from the kernels. It would be nice to have truly shared data structures like CUDA gives you in C++!

even putting aside how rust ownership semantics map poorly onto gpu programming, ml researchers will never learn rust, this will never ever happen...

The ML researchers are relying on libraries written by someone else. Today, those libraries are mostly C++, and they would benefit from Rust same as most other C++ codebases.

Re: Nvidia adds native Python support to CUDA

#122

Post by gavinray »

nickysielicki wrote: post #112
bee_rider wrote: post #107

Earlier quoted context omitted.

If I have a mostly CPU code and I want to time the scenario: “I have just a couple subroutines that I am willing to offload to the GPU,” what’s wrong with sprinkling my code with normal old python timing calls? If I don’t care what part of the CUDA ecosystem is taking time (from my point of view it is a black-box that does GEMMs) so why not measure “time until my normal code is running again?”

If you care enough to time it, you should care enough to time it correctly.

One of the wisest things I've read all week.

I authored one of the primary tools for GraphQL server benchmarks.

I learned about the Coordinated Omission problem and formats like HDR Histograms during the implementation.

My takeaway from that project is that not only is benchmarking anything correctly difficult, but they all ought to come with disclaimers of:

"These are the results obtained on X machine, running at Y time, with Z resources."

Re: Nvidia adds native Python support to CUDA

#123

Post by int_19h »

chrisrodrigue wrote: post #47

Python is really shaping up to be the lingua franca of programming languages. Its adoption is soaring in this FOSS renaissance and I think it's the closest thing to a golden hammer that we've ever had. The PEP model is a good vehicle for self-improvement and standardization. Packaging and deployment will soon be solved problems thanks to projects such as uv and BeeWare, and I'm confident that we're going to see conti…

AI-generated code is going to be a major influence going forward. Regardless of how you feel about its quality (I'm a pessimist myself), it's happening anyway, and it's going to cement the dominant position of those languages which LLMs understand / can write the best. Which correlates strongly to their amount in the training set, which means that Python and JavaScript in particular are here to stay now, and will likely be increasingly shoved into more and more niches - even those they aren't well-suited to - solely because LLMs can write them.

Re: Nvidia adds native Python support to CUDA

#124

Post by airstrike »

system2 wrote: post #118
airstrike wrote: post #76

Earlier quoted context omitted.

As someone who spent nearly a decade with Python, I'd say 90% of people will answer "yes", so I'd like to offer a different perspective. IMHO if you want to pick it up for a couple toy projects just to get a feel of what coding is like, then by all means try it out. But eventually you'll benefit tremendously from exploring other languages. Python will teach you a lot of bad habits. You will feel like you know what yo…

With the help of GPT, I think the bad habit part is non-existent anymore. Learning it from GPT really helps people nowadays. Ask ChatGPT 4.0 some questions, and you will be shocked by how well it describes the code. Just don't ask to fix indentations because it will do it line by line for hours. But it finds mistakes quickly and points you in the right direction. And of course, it comes up with random non-existent mo…

The bad habits I was thinking about were more in the line of not understanding how memory is being used (even something as simple as stack vs. heap allocation), not having a type system that forces you to think about the types of data structure you have in your system, and overall just being forced to design before coding

Re: Nvidia adds native Python support to CUDA

#125

Post by ryao »

pjmlp wrote: post #97
ryao wrote: post #57

CUDA was born from C and C++ It would be nice if they actually implemented a C variant of CUDA instead of extending C++ and calling it CUDA C.

First of all they extend C, and with CUDA 3.0, initial support was added for C++, afterwards they bought PGI and added Fortran into the mix. Alongside for the ride, they fostered an ecosystem from compiled language backends targeting CUDA. Additionally modern CUDA supports standard C++ as well, with frameworks that hide the original extensions. Most critics don't really get the CUDA ecosystem.

They replaced C with C++. For example, try passing a function pointer as a void pointer argument without a cast. C says this should work. C++ says it should not. There are plenty of other differences that make it C++ and not C, if you know to look for them. The fact that C++ symbol names are used for one, which means you need to specify extern “C” if you want to reference them from the CUDA driver API. Then there is the fact that it will happily compile C++ classes where a pure C compiler will not. There is no stdc option for the compiler.

Re: Nvidia adds native Python support to CUDA

#126

Post by hackernudes »

ImprobableTruth wrote: post #103
claytonjy wrote: post #79

Earlier quoted context omitted.

I’ve always thought it was weird GPU stuff in python doesn’t use asyncio, and mostly assumed it was because python-on-GPU predates asyncio. But I was hoping a new lib like this might right that wrong, but it doesn’t. Maybe for interop reasons? Do other languages surface the asynchronous nature of GPUs in language-level async, avoiding silly stuff like synchronize?

The reason is that the usage is completely different from coroutine based async. With GPUs you want to queue _as many async operations as possible_ and only then synchronize. That is, you would have a program like this (pseudocode): b = foo(a) c = bar(b) d = baz(c) synchronize() With coroutines/async await, something like this b = await foo(a) c = await bar(b) d = await baz(c) would synchronize after every step, bein…

Pretty sure you want it to do it the first way in all cases (not just with GPUs)!

Re: Nvidia adds native Python support to CUDA

#127

Post by ryao »

swyx wrote: post #67
ryao wrote: post #57

CUDA was born from C and C++ It would be nice if they actually implemented a C variant of CUDA instead of extending C++ and calling it CUDA C.

why is that impt to you? just trying to understand the problem you couldnt solve without a C-like

I want to write C code, not C++ code. Even if I try to write C style C++, it is more verbose and less readable, because of various C++isms. For example, having to specify extern “C” to get sane ABI names for the Nvidia CUDA driver API:

https://docs.nvidia.com/cuda/cuda-driver-api/index.html

Not to mention that C++ does not support neat features like variable sized arrays on the stack.

Re: Nvidia adds native Python support to CUDA

#128

Post by screye »

chrisrodrigue wrote: post #47

Python is really shaping up to be the lingua franca of programming languages. Its adoption is soaring in this FOSS renaissance and I think it's the closest thing to a golden hammer that we've ever had. The PEP model is a good vehicle for self-improvement and standardization. Packaging and deployment will soon be solved problems thanks to projects such as uv and BeeWare, and I'm confident that we're going to see conti…

Never heard of Beeware, but Astral's products have transformed my python workflow (uv, ruff).

Is Beeware that transformational ? What does Beeware do and what is its maturity level?

Re: Nvidia adds native Python support to CUDA

#129

Post by crazygringo »

Very curious how this compares to JAX [1].

JAX lets you write Python code that executes on Nvidia, but also GPUs of other brands (support varies). It similarly has drop-in replacements for NumPy functions.

This only supports Nvidia. But can it do things JAX can't? It is easier to use? Is it less fixed-size-array-oriented? Is it worth locking yourself into one brand of GPU?

[1] https://github.com/jax-ml/jax

Post reply on HN