Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia adds native Python support to CUDA

thenewstack.io

Post reply on HN
Print view View original

51–60 of 196 posts

Re: Nvidia adds native Python support to CUDA

#51

Post by dismalaf »

gymbeaux wrote: post #12

This is huge. Anyone who was considering AMD + ROCm as an alternative to NVIDIA in the AI space isn’t anymore. I’m one of those people who can’t (won’t) learn C++ to the extent required to effectively write code for GPU execution…. But to have a direct pipeline to the GPU via Python. Wow. The efficiency implications are huge, not just for Python libraries like PyTorch, but also anything we write that runs on an NVIDI…

> But to have a direct pipeline to the GPU via Python

Have you ever used a GPU API (CUDA, OpenCL, OpenGL, Vulkan, etc...) with a scripting language?

It's cool that Nvidia made a bit of an ecosystem around it but it won't replace C++ or Fortran and you can't simply drop in "normal" Python code and have it run on the GPU. CUDA is still fundamentally it's own thing.

There's also been CUDA bindings to scripting languages for at least 15 years... Most people will probably still use Torch or higher level things built on top of it.

Also, here's Nvidia's own advertisement and some instructions for Python on their GPUs:

- https://developer.nvidia.com/cuda-python

- https://developer.nvidia.com/how-to-cuda-python

Reality is kind of boring, and the article posted here is just clickbait.

Re: Nvidia adds native Python support to CUDA

#52

Post by whycome »

chrisrodrigue wrote: post #47

Python is really shaping up to be the lingua franca of programming languages. Its adoption is soaring in this FOSS renaissance and I think it's the closest thing to a golden hammer that we've ever had. The PEP model is a good vehicle for self-improvement and standardization. Packaging and deployment will soon be solved problems thanks to projects such as uv and BeeWare, and I'm confident that we're going to see conti…

Would you say Python is a good language to learn as a beginner?

Re: Nvidia adds native Python support to CUDA

#53

Post by moffkalast »

diggan wrote: post #5

I'm no GPU programmer, but seems easy to use even for someone like me. I pulled together a quick demo of using the GPU vs the CPU, based on what I could find ( https://gist.github.com/victorb/452a55dbcf59b3cbf84efd8c3097... ) which gave these results (after downloading 2.6GB of dependencies of course): Creating 100 random matrices of size 5000x5000 on CPU... Adding matrices using CPU... CPU matrix addition completed…

Only 4x speed seems rather low for GPU acceleration, does numpy already use AVX2 or anything SIMD?

For comparison, doing something similar with torch on CPU and torch on GPU will get you like 100x speed difference.

Re: Nvidia adds native Python support to CUDA

#54

Post by dpkirchner »

whycome wrote: post #52
chrisrodrigue wrote: post #47

Python is really shaping up to be the lingua franca of programming languages. Its adoption is soaring in this FOSS renaissance and I think it's the closest thing to a golden hammer that we've ever had. The PEP model is a good vehicle for self-improvement and standardization. Packaging and deployment will soon be solved problems thanks to projects such as uv and BeeWare, and I'm confident that we're going to see conti…

Would you say Python is a good language to learn as a beginner?

Not the person you replied to but I'd say definitely not. It'd be easy to pick up bad habits from python (untyped variables) and try to carry them over to other languages. It's also the king of runtime errors, which will frustrate newbies.

I think a compiled language is a better choice for people just getting started. Java is good, IMO, because it is verbose. Eventually the beginner may get tired of the verbosity and move on to something else, but at least they'll understand the value of explicit types and compile-time errors.

Re: Nvidia adds native Python support to CUDA

#55

Post by chasely »

the__alchemist wrote: post #28
chasely wrote: post #20

Earlier quoted context omitted.

The Rust-CUDA project just recently started up again [0], I've started digging into it a little bit and am hoping to contribute to it since the summers are a little slower for me. [0] https://github.com/rust-gpu/rust-cuda

Still broken though! Has been for years. In a recent GH issue regarding desires for the reboot, I asked: "Try it on a few different machines (OS, GPUs, CUDA versions etc), make it work on modern RustC and CUDA versions without errors." The response was "That will be quite some work." Meanwhile, Cudarc works...

Totally, it's going to take a minute to get it all working. On a positive note, they recently got some sponsorship from Modal [0], who is supplying GPUs for CI/CD so they should be able to expand their hardware coverage.

Re: Nvidia adds native Python support to CUDA

#56

Post by malcolmgreaves »

taminka wrote: post #31
the__alchemist wrote: post #8

Rust support next? RN I am manually [de]serializing my data structures as byte arrays to/from the kernels. It would be nice to have truly shared data structures like CUDA gives you in C++!

even putting aside how rust ownership semantics map poorly onto gpu programming, ml researchers will never learn rust, this will never ever happen...

ML reachers don’t write code, they ask ChatGPT to make a horribly inefficient, non-portable notebook that has to be rewritten from scratch :)

Re: Nvidia adds native Python support to CUDA

#58

Post by diggan »

moffkalast wrote: post #53
diggan wrote: post #5

I'm no GPU programmer, but seems easy to use even for someone like me. I pulled together a quick demo of using the GPU vs the CPU, based on what I could find ( https://gist.github.com/victorb/452a55dbcf59b3cbf84efd8c3097... ) which gave these results (after downloading 2.6GB of dependencies of course): Creating 100 random matrices of size 5000x5000 on CPU... Adding matrices using CPU... CPU matrix addition completed…

Only 4x speed seems rather low for GPU acceleration, does numpy already use AVX2 or anything SIMD? For comparison, doing something similar with torch on CPU and torch on GPU will get you like 100x speed difference.

It's a microbenchmark (if even that), take it with a grain of salt. You'd probably see a bigger difference with bigger/more/more complicated tasks,

Re: Nvidia adds native Python support to CUDA

#60

Post by hbn »

DeathArrow wrote: post #18

>In 2024, Python became the most popular programming language in the world — overtaking JavaScript — according to GitHub’s 2024 open source survey. I wonder why Python take over the world? Of course, it's easy to learn, it might be easy to read and understand. But it also has a few downsides: low performance, single threaded, lack of static typing.

It's "easy to learn" and you get all the downsides that come with that.

At work right now we're integrating with scoring models hosted in Amazon SageMaker written by a "modelling team" and as far as I can tell they follow absolutely no basic coding practices. They give us the API and are asking us to send English strings of text for names of things instead of any real keys, and they're just comparing against plain strings and magic numbers everywhere so if they're asked to make any change like renaming something it's a herculean task that breaks a bunch of other things. Something will break when a field is null and then they'll tell us instead of sending null if we have no data to send -9999999. One time something broke and it turned out to be because we sent them "MB" (Manitoba) as someone's province, and whoever wrote it was just plain-text checking against a list of province codes as strings and didn't remember to include Manitoba.

I know this is still mainly a business/management issue that they're allowing people who don't know how to code to write code, but I'm sure this is happening at other companies, and I think Python's level of accessibility at the beginner level has been a real blight to software quality.

Post reply on HN