Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia adds native Python support to CUDA

thenewstack.io

Post reply on HN
Print view View original

81–90 of 196 posts

Re: Nvidia adds native Python support to CUDA

#81

Post by airstrike »

chrisrodrigue wrote: post #47

Python is really shaping up to be the lingua franca of programming languages. Its adoption is soaring in this FOSS renaissance and I think it's the closest thing to a golden hammer that we've ever had. The PEP model is a good vehicle for self-improvement and standardization. Packaging and deployment will soon be solved problems thanks to projects such as uv and BeeWare, and I'm confident that we're going to see conti…

Python is too high level, slow and duck-typed to even be considered for a huge number of projects.

There is no one-size-fits-all programming language.

Re: Nvidia adds native Python support to CUDA

#82

Post by apbytes »

claytonjy wrote: post #79
apbytes wrote: post #46

Earlier quoted context omitted.

When you call a cuda method, it is launched asynchronously. That is the function queues it up for execution on gpu and returns. So if you need to wait for an op to finish, you need to `synchronize` as shown above. `get_current_stream` because the queue mentioned above is actually called stream in cuda. If you want to run many independent ops concurrently, you can use several streams. Benchmarking is one use case for…

I’ve always thought it was weird GPU stuff in python doesn’t use asyncio, and mostly assumed it was because python-on-GPU predates asyncio. But I was hoping a new lib like this might right that wrong, but it doesn’t. Maybe for interop reasons? Do other languages surface the asynchronous nature of GPUs in language-level async, avoiding silly stuff like synchronize?

Might have to look at specific lib implementations, but I'd guess that mostly gpu calls from python are actually happening in c++ land. And internally a lib might be using synchronize calls where needed.

Re: Nvidia adds native Python support to CUDA

#83

Post by staunton »

malcolmgreaves wrote: post #56
taminka wrote: post #31

Earlier quoted context omitted.

even putting aside how rust ownership semantics map poorly onto gpu programming, ml researchers will never learn rust, this will never ever happen...

ML reachers don’t write code, they ask ChatGPT to make a horribly inefficient, non-portable notebook that has to be rewritten from scratch :)

It's made easier by that notebook only having to work just once, to produce some plots for the paper/press release/demo.

Re: Nvidia adds native Python support to CUDA

#84

Post by taminka »

pjmlp wrote: post #36
taminka wrote: post #31

Earlier quoted context omitted.

even putting aside how rust ownership semantics map poorly onto gpu programming, ml researchers will never learn rust, this will never ever happen...

While I agree in principle, CUDA is more than only AI, as people keep forgetting.

everyone else who uses cuda isn't going to learn rust either

Re: Nvidia adds native Python support to CUDA

#85

Post by yieldcrv »

almostgotcaught wrote: post #72
ashvardanian wrote: post #48

Earlier quoted context omitted.

CuPy has been available for years and has always worked great. The article is about the next wave of Python-oriented JIT toolchains, that will allow writing actual GPU kernels in a Pythonic-style instead of calling an existing precompiled GEMM implementation in CuPy (like in that snippet) or even JIT-ing CUDA C++ kernels from a Python source, that has also been available for years: https://docs.cupy.dev/en/stable/use…

it's funny - people around here really do not have a clue about the GPU ecosystem even though everyone is always talking about AI: > The article is about the next wave of Python-oriented JIT toolchains the article is content marketing (for whatever) but the actual product has literally has nothing to do with kernels or jitting or anything https://github.com/NVIDIA/cuda-python literally just cython bindings to CUDA ru…

I just want to see benchmarks. is this new one faster than CuPy or not

Re: Nvidia adds native Python support to CUDA

#86

Post by ashvardanian »

almostgotcaught wrote: post #72
ashvardanian wrote: post #48

Earlier quoted context omitted.

CuPy has been available for years and has always worked great. The article is about the next wave of Python-oriented JIT toolchains, that will allow writing actual GPU kernels in a Pythonic-style instead of calling an existing precompiled GEMM implementation in CuPy (like in that snippet) or even JIT-ing CUDA C++ kernels from a Python source, that has also been available for years: https://docs.cupy.dev/en/stable/use…

it's funny - people around here really do not have a clue about the GPU ecosystem even though everyone is always talking about AI: > The article is about the next wave of Python-oriented JIT toolchains the article is content marketing (for whatever) but the actual product has literally has nothing to do with kernels or jitting or anything https://github.com/NVIDIA/cuda-python literally just cython bindings to CUDA ru…

In case someone is looking for some performance examples & testimonials, even on RTX 3090 vs a 64-core AMD Epy/Threadripper, even a couple of years ago, CuPy was a blast. I have a couple of recorded sessions with roughly identical slides/numbers:

  - San Francisco Python meetup in 2023: https://youtu.be/L9ELuU3GeNc?si=TOp8lARr7rP4cYaw
  - Yerevan PyData meetup in 2022: https://youtu.be/OxAKSVuW2Yk?si=5s_G0hm7FvFHXx0u
Of the more remarkable results:

  - 1000x sorting speedup switching from NumPy to CuPy.
  - 50x performance improvements switching from Pandas to CuDF on the New York Taxi Rides queries.
  - 20x GEMM speedup switching from NumPy to CuPy.
CuGraph is also definitely worth checking out. At that time, Intel wasn't in as bad of a position as they are now and was trying to push Modin, but the difference in performance and quality of implementation was mind-boggling.

Re: Nvidia adds native Python support to CUDA

#87

Post by wafngar »

disgruntledphd2 wrote: post #17
aixpert wrote: post #6

thank God, Pytorch gained so much momentum before this came out, Now we have a true platform independent semi standard For parallel computations. We are not stuck with NVIDIA specifics. It's great that parts of pie torch which concern the NVIDIA backend can now be implemented in Python directly, The important part that it doesn't really matter or shouldn't matter for end users / Developers that being said, maybe this…

This just makes it much, much easier for people to build numeric stuff on GPU, which is great. I'm totally with you that it's better that this took so long, so we have things like PyTorch abstracting most of this away, but I'm looking forward to (in my non-existent free time :/ ) playing with this.

Why not use torch.compile()?

Re: Nvidia adds native Python support to CUDA

#90

Post by ashvardanian »

CuTile, in many ways, feels like a successor to OpenAI's Triton... And not only are we getting tile/block-level primitives and TileIR, but also a proper SIMT programming model in CuPy, which I don't think enough people noticed even at this year's GTC. Very cool stuff!

That said, there were almost no announcements or talks related to CPUs, despite the Grace CPUs being announced quite some time ago. It doesn't feel like we're going to see generalizable abstractions that work seamlessly across Nvidia CPUs and GPUs anytime soon. For someone working on parallel algorithms daily, this is an issue: debugging with NSight and CUDA-GDB still isn't the same as raw GDB, and it's much easier to design algorithms on CPUs first and then port them to GPUs.

Of all the teams in the compiler space, Modular seems to be among the few that aren't entirely consumed by the LLM craze, actively building abstractions and languages spanning multiple platforms. Given the landscape, that's increasingly valuable. I'd love to see more people experimenting with Mojo — perhaps it can finally bridge the CPU-GPU gap that many of us face daily!

Post reply on HN