Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Nvidia adds native Python support to CUDA

thenewstack.io

Post reply on HN
Print view View original

101–110 of 196 posts

Re: Nvidia adds native Python support to CUDA

#101

Post by lenkite »

DeathArrow wrote: post #18

>In 2024, Python became the most popular programming language in the world — overtaking JavaScript — according to GitHub’s 2024 open source survey. I wonder why Python take over the world? Of course, it's easy to learn, it might be easy to read and understand. But it also has a few downsides: low performance, single threaded, lack of static typing.

> I wonder why Python take over the world

Because data-science/ML/LLM's have taken over the world now and no other language offers best-in-breed libraries and frameworks.

Other languages need to get off their ass and start offering options soon or be relegated to niche domains.

Re: Nvidia adds native Python support to CUDA

#102

Post by nickysielicki »

wiredfool wrote: post #15
diggan wrote: post #5

I'm no GPU programmer, but seems easy to use even for someone like me. I pulled together a quick demo of using the GPU vs the CPU, based on what I could find ( https://gist.github.com/victorb/452a55dbcf59b3cbf84efd8c3097... ) which gave these results (after downloading 2.6GB of dependencies of course): Creating 100 random matrices of size 5000x5000 on CPU... Adding matrices using CPU... CPU matrix addition completed…

Curious what the timing would be if it included the memory transfer time, e.g. matricies = [np.random(...) for _ in range] time_start = time.time() cp_matricies = [cp.array(m) for m in matrices] add_(cp_matricies) sync time_end = time.time()

I don’t mean to call you or your pseudocode out specifically, but I see this sort of thing all the time, and I just want to put it out there:

PSA: if you ever see code trying to measure timing and it’s not using the CUDA event APIs, it’s fundamentally wrong and is lying to you. The simplest way to be sure you’re not measuring noise is to just ban the usage of any other timing source. Definitely don’t add unnecessary syncs just so that you can add a timing tap.

https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART_...

Re: Nvidia adds native Python support to CUDA

#103

Post by ImprobableTruth »

claytonjy wrote: post #79
apbytes wrote: post #46

Earlier quoted context omitted.

When you call a cuda method, it is launched asynchronously. That is the function queues it up for execution on gpu and returns. So if you need to wait for an op to finish, you need to `synchronize` as shown above. `get_current_stream` because the queue mentioned above is actually called stream in cuda. If you want to run many independent ops concurrently, you can use several streams. Benchmarking is one use case for…

I’ve always thought it was weird GPU stuff in python doesn’t use asyncio, and mostly assumed it was because python-on-GPU predates asyncio. But I was hoping a new lib like this might right that wrong, but it doesn’t. Maybe for interop reasons? Do other languages surface the asynchronous nature of GPUs in language-level async, avoiding silly stuff like synchronize?

The reason is that the usage is completely different from coroutine based async. With GPUs you want to queue _as many async operations as possible_ and only then synchronize. That is, you would have a program like this (pseudocode):

  b = foo(a)
  c = bar(b)
  d = baz(c)
  synchronize()
With coroutines/async await, something like this

  b = await foo(a)
  c = await bar(b)
  d = await baz(c)
would synchronize after every step, being much more inefficient.

Re: Nvidia adds native Python support to CUDA

#104

Post by rahimnathwani »

rahimnathwani wrote: post #10
diggan wrote: post #5

I'm no GPU programmer, but seems easy to use even for someone like me. I pulled together a quick demo of using the GPU vs the CPU, based on what I could find ( https://gist.github.com/victorb/452a55dbcf59b3cbf84efd8c3097... ) which gave these results (after downloading 2.6GB of dependencies of course): Creating 100 random matrices of size 5000x5000 on CPU... Adding matrices using CPU... CPU matrix addition completed…

Thank you. I scrolled up and down the article hoping they included a code sample.

EDIT: Just realized the code doesn't seem to be using the GPU for the addition.

Re: Nvidia adds native Python support to CUDA

#105

Post by bigyabai »

pjmlp wrote: post #92
bigyabai wrote: post #74

Earlier quoted context omitted.

It can be argued that they already did. AMD and Apple worked with Khronos to build OpenCL as a general competitor. The industry didn't come together to support it though, and eventually major stakeholders abandoned it altogether. Those ~10 wasted years were spent on Nvidia's side refining their software offerings and redesigning their GPU architecture to prioritize AI performance over raster optimization. Meanwhile A…

Maybe because Apple got pissed on how Khronos took over OpenCL, AMD and Intel never offered tooling on par with CUDA in terms of IDE integration, graphical debuggers and library ecosystem. Khronos also never saw the need to support a polyglot ecosystem with C++, Fortran and anything else that the industry could feel like using on a GPU. When Khronos finally remember to at least add C++ support and SPIR, again Intel a…

Well, Apple has done nothing to replace the common standard they abandoned. They failed to develop their proprietary alternatives into a competitive position and now can't even use their own TSMC dies (imported at great expense) for training: https://www.eteknix.com/apple-set-to-invest-1-billion-in-nvi...

However you want to paint the picture today, you can't say the industry didn't try to resist CUDA. The stakeholders shot each other in a 4-way Mexican standoff, and Nvidia whistled showtunes all the way to the bank. If OpenCL was treated with the same importance Vulkan was, we might see a very different market today.

Re: Nvidia adds native Python support to CUDA

#106

Post by kevmo314 »

swyx wrote: post #67
ryao wrote: post #57

CUDA was born from C and C++ It would be nice if they actually implemented a C variant of CUDA instead of extending C++ and calling it CUDA C.

why is that impt to you? just trying to understand the problem you couldnt solve without a C-like

A strict C variant would indeed be quite nice. I've wanted to write CUDA kernels in Go apps before so the Go app can handle the concurrency on the CPU side. Right now, I have to write a C wrapper and more often than not, I end up writing more code in C++ instead.

But then I end up finding myself juggling mutexes and wishing I had some newer language features.

Re: Nvidia adds native Python support to CUDA

#107

Post by bee_rider »

nickysielicki wrote: post #102
wiredfool wrote: post #15

Earlier quoted context omitted.

Curious what the timing would be if it included the memory transfer time, e.g. matricies = [np.random(...) for _ in range] time_start = time.time() cp_matricies = [cp.array(m) for m in matrices] add_(cp_matricies) sync time_end = time.time()

I don’t mean to call you or your pseudocode out specifically, but I see this sort of thing all the time, and I just want to put it out there: PSA: if you ever see code trying to measure timing and it’s not using the CUDA event APIs, it’s fundamentally wrong and is lying to you. The simplest way to be sure you’re not measuring noise is to just ban the usage of any other timing source. Definitely don’t add unnecessary…

If I have a mostly CPU code and I want to time the scenario: “I have just a couple subroutines that I am willing to offload to the GPU,” what’s wrong with sprinkling my code with normal old python timing calls?

If I don’t care what part of the CUDA ecosystem is taking time (from my point of view it is a black-box that does GEMMs) so why not measure “time until my normal code is running again?”

Re: Nvidia adds native Python support to CUDA

#108

Post by pjmlp »

bigyabai wrote: post #105
pjmlp wrote: post #92

Earlier quoted context omitted.

Maybe because Apple got pissed on how Khronos took over OpenCL, AMD and Intel never offered tooling on par with CUDA in terms of IDE integration, graphical debuggers and library ecosystem. Khronos also never saw the need to support a polyglot ecosystem with C++, Fortran and anything else that the industry could feel like using on a GPU. When Khronos finally remember to at least add C++ support and SPIR, again Intel a…

Well, Apple has done nothing to replace the common standard they abandoned. They failed to develop their proprietary alternatives into a competitive position and now can't even use their own TSMC dies (imported at great expense) for training: https://www.eteknix.com/apple-set-to-invest-1-billion-in-nvi... However you want to paint the picture today, you can't say the industry didn't try to resist CUDA. The stakeholde…

Yes they did, it is called Metal Compute, and everyone using Apple devices has to use it.

Vulkan you say?

It is only relevant on GNU/Linux and Android, because Google is pushing it, and still most folks still keep using OpenGL ES, no one else cares about it, and already turned into the same spaghetti mess as OpenGL, to the point that there was a roadmap talk at Vulkanised 2025 on how to sort things out.

NVidia and AMD keep designing their cards with Microsoft for DirectX first, and Vulkan, eventually.

Re: Nvidia adds native Python support to CUDA

#109

Post by IshKebab »

KeplerBoy wrote: post #45
the__alchemist wrote: post #8

Rust support next? RN I am manually [de]serializing my data structures as byte arrays to/from the kernels. It would be nice to have truly shared data structures like CUDA gives you in C++!

Isn't Rust still very seldomly used in the areas where CUDA shines (e.g. number crunching of any kind, let it be simulations or linear algebra)? Imo C++ or even Fortran are perfectly fine choices for those things, since the memory allocation pattern aren't that complicated.

Mainly because number crunching code tends to be very long-lived (hence why FORTRAN is still in use).

Re: Nvidia adds native Python support to CUDA

#110

Post by bigyabai »

pjmlp wrote: post #108
bigyabai wrote: post #105

Earlier quoted context omitted.

Well, Apple has done nothing to replace the common standard they abandoned. They failed to develop their proprietary alternatives into a competitive position and now can't even use their own TSMC dies (imported at great expense) for training: https://www.eteknix.com/apple-set-to-invest-1-billion-in-nvi... However you want to paint the picture today, you can't say the industry didn't try to resist CUDA. The stakeholde…

Yes they did, it is called Metal Compute, and everyone using Apple devices has to use it. Vulkan you say? It is only relevant on GNU/Linux and Android, because Google is pushing it, and still most folks still keep using OpenGL ES, no one else cares about it, and already turned into the same spaghetti mess as OpenGL, to the point that there was a roadmap talk at Vulkanised 2025 on how to sort things out. NVidia and AM…

> it is called Metal Compute, and everyone using Apple devices has to use it.

Sounds like a submarket absolutely teeming with competition. Like, you have Metal Compute, and Apple Accelerate Framework and MLX all sitting there in the same spot! Apple is really outdoing themselves, albeit in a fairly literal sense.

> It is only relevant on GNU/Linux and Android

Hmm... someone ought to remind me of the first stage of grief, I've forgotten it suddenly.

Post reply on HN