Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia adds native Python support to CUDA

thenewstack.io

Post reply on HN
Print view View original

61–70 of 196 posts

Re: Nvidia adds native Python support to CUDA

#61

Post by chrisrodrigue »

whycome wrote: post #52
chrisrodrigue wrote: post #47

Python is really shaping up to be the lingua franca of programming languages. Its adoption is soaring in this FOSS renaissance and I think it's the closest thing to a golden hammer that we've ever had. The PEP model is a good vehicle for self-improvement and standardization. Packaging and deployment will soon be solved problems thanks to projects such as uv and BeeWare, and I'm confident that we're going to see conti…

Would you say Python is a good language to learn as a beginner?

Yeah, definitely. It's basically executable pseudocode and it's really simple for a beginner to pick up and hit the ground running for a variety of use cases.

Some people will tell you to start with C or C++ to get a better intuition for what's actually happening under the hood in Python, but that's not really necessary for most use cases unless you're doing something niche. Some of the most popular use cases for Python are webapps, data analysis, or general automation. For the 1% of use cases that Python isn't the right fit for, you can still use it to prototype or glue things together.

There are a lot of great resources out there for learning Python, but they won't necessarily teach you how to make great software. You can't go wrong with the official tutorial. https://learn.scientific-python.org/development/ is pretty terse and incorporates a lot of best practices.

Re: Nvidia adds native Python support to CUDA

#62

Post by freeone3000 »

dismalaf wrote: post #51
gymbeaux wrote: post #12

This is huge. Anyone who was considering AMD + ROCm as an alternative to NVIDIA in the AI space isn’t anymore. I’m one of those people who can’t (won’t) learn C++ to the extent required to effectively write code for GPU execution…. But to have a direct pipeline to the GPU via Python. Wow. The efficiency implications are huge, not just for Python libraries like PyTorch, but also anything we write that runs on an NVIDI…

> But to have a direct pipeline to the GPU via Python Have you ever used a GPU API (CUDA, OpenCL, OpenGL, Vulkan, etc...) with a scripting language? It's cool that Nvidia made a bit of an ecosystem around it but it won't replace C++ or Fortran and you can't simply drop in "normal" Python code and have it run on the GPU. CUDA is still fundamentally it's own thing. There's also been CUDA bindings to scripting languages…

OpenCL and OpenGL are basically already scripting languages that you happen to type into a C compiler. The CUDA advantage was actually having meaningful types and compilation errors, without the intense boilerplate of Vulkan. But this is 100% a python-for-CUDA-C replacement on the GPU, for people who prefer a slightly different bracketing syntax.

Re: Nvidia adds native Python support to CUDA

#63

Post by hnuser123456 »

apbytes wrote: post #46
hnuser123456 wrote: post #23

Earlier quoted context omitted.

I think it does?: (the comment is in the original source) print("Adding matrices using GPU...") start_time = time.time() gpu_result = add_matrices(gpu_matrices) cp.cuda.get_current_stream().synchronize() # Not 100% sure what this does elapsed_time = time.time() - start_time I was going to ask, any CUDA professionals who want to give a crash course on what us python guys will need to know?

When you call a cuda method, it is launched asynchronously. That is the function queues it up for execution on gpu and returns. So if you need to wait for an op to finish, you need to `synchronize` as shown above. `get_current_stream` because the queue mentioned above is actually called stream in cuda. If you want to run many independent ops concurrently, you can use several streams. Benchmarking is one use case for…

Thank you kindly!

Re: Nvidia adds native Python support to CUDA

#64

Post by horsawlarway »

silisili wrote: post #49
chrisrodrigue wrote: post #47

Python is really shaping up to be the lingua franca of programming languages. Its adoption is soaring in this FOSS renaissance and I think it's the closest thing to a golden hammer that we've ever had. The PEP model is a good vehicle for self-improvement and standardization. Packaging and deployment will soon be solved problems thanks to projects such as uv and BeeWare, and I'm confident that we're going to see conti…

> Packaging and deployment will soon be solved problems I really hope you're right. I love Python as a language, but for any sufficiently large project, those items become an absolute nightmare without something like Docker. And even with, there seems to be multiple ways people solve it. I wish they'd put something in at the language level or bless an 'official' one. Go has spoiled me there.

Honestly, I'm still incredibly shocked at just how bad Python is on this front.

I'm plenty familiar with packaging solutions that are painful to work with, but the state of python was shocking when I hopped back in because of the available ML tooling.

UV seems to be at least somewhat better, but damn - watching pip literally download 20+ 800MB torch wheels over and over trying to resolve deps only to waste 25GB of bandwidth and finally completely fail after taking nearly an hour was absolutely staggering.

Re: Nvidia adds native Python support to CUDA

#65

Post by andrewmcwatters »

CapsAdmin wrote: post #19

Slightly related, I had a go at doing llama 3 inference in luajit using cuda as one compute backend for just doing matrix multiplication https://github.com/CapsAdmin/luajit-llama3/blob/main/compute... While obviously not complete, it was less than I thought was needed. It was a bit annoying trying to figure out which version of the function (_v2 suffix) I have to use for which driver I was running. Also sometimes a b…

Neat, thanks for sharing!

Re: Nvidia adds native Python support to CUDA

#68

Post by dragonwriter »

dismalaf wrote: post #51
gymbeaux wrote: post #12

This is huge. Anyone who was considering AMD + ROCm as an alternative to NVIDIA in the AI space isn’t anymore. I’m one of those people who can’t (won’t) learn C++ to the extent required to effectively write code for GPU execution…. But to have a direct pipeline to the GPU via Python. Wow. The efficiency implications are huge, not just for Python libraries like PyTorch, but also anything we write that runs on an NVIDI…

> But to have a direct pipeline to the GPU via Python Have you ever used a GPU API (CUDA, OpenCL, OpenGL, Vulkan, etc...) with a scripting language? It's cool that Nvidia made a bit of an ecosystem around it but it won't replace C++ or Fortran and you can't simply drop in "normal" Python code and have it run on the GPU. CUDA is still fundamentally it's own thing. There's also been CUDA bindings to scripting languages…

> It's cool that Nvidia made a bit of an ecosystem around it but it won't replace C++ or Fortran and you can't simply drop in "normal" Python code and have it run on the GPU.

While its not exactly normal Python code, there are Python libraries that allow writing GPU kernels in internal DSLs that are normal-ish Python (e.g., Numba for CUDA specifically via the @cuda.jit decorator; or Taichi which has multiple backends supporting the same application code—Vulkan, Metal, CUDA, OpenGL, OpenGL ES, and CPU.)

Apparently, nVidia is now doing this first party in CUDA Python, including adding a new paradigm for CUDA code (CuTile) that is going to be in Python before C++; possibly trying to get ahead of things like Taichi (which, because it is cross-platform, commoditizes the underlying GPU).

> Also, here's Nvidia's own advertisement for Python on their GPUs

That (and the documentation linked there) does not address the new upcoming native functionality announced at GTC; existing CUDA Python has kernels written in C++ in inline strings.

Re: Nvidia adds native Python support to CUDA

#69

Post by dismalaf »

freeone3000 wrote: post #62
dismalaf wrote: post #51

Earlier quoted context omitted.

> But to have a direct pipeline to the GPU via Python Have you ever used a GPU API (CUDA, OpenCL, OpenGL, Vulkan, etc...) with a scripting language? It's cool that Nvidia made a bit of an ecosystem around it but it won't replace C++ or Fortran and you can't simply drop in "normal" Python code and have it run on the GPU. CUDA is still fundamentally it's own thing. There's also been CUDA bindings to scripting languages…

OpenCL and OpenGL are basically already scripting languages that you happen to type into a C compiler. The CUDA advantage was actually having meaningful types and compilation errors, without the intense boilerplate of Vulkan. But this is 100% a python-for-CUDA-C replacement on the GPU, for people who prefer a slightly different bracketing syntax.

> But this is 100% a python-for-CUDA-C replacement on the GPU

Ish. It's a Python maths library made by Nvidia, an eDSL and a collection of curated libraries. It's not significantly different than stuff like Numpy, Triton, etc..., apart from being made by Nvidia and bundled with their tools.

Post reply on HN