Unifying the CUDA Python Ecosystem • Hacker News Board

Re: Unifying the CUDA Python Ecosystem

#21

Post by jjoonathan » Fri, Apr 16, 2021, 5:23 PM UTC

andrew_v4 wrote: post #7

Just for contrast its interesting to look at an example of writing a similar kernel in Julia: https://juliagpu.gitlab.io/CUDA.jl/tutorials/introduction/ I don't think it's possible to achieve something like this in python because of how it's interpreted (but it sounds a bit like what another comment mentioned where the python was compiled to C)

> Julia has first-class support for GPU programming

"First-class" is a steep claim. Does it support the nvidia perf tools? Those are very important for taking a kernel from (in my experience) ~20% theoretical perf to ~90% theoretical perf.

Re: Unifying the CUDA Python Ecosystem

#22

Post by dwrodri » Fri, Apr 16, 2021, 5:28 PM UTC

albertzeyer wrote: post #5

michelpp wrote: post #2

About 8 years ago an NVIDIA developer released a tool called Copperhead that let you write CUDA kernels in straight Python that were then compiled to C, no "C-in-a-string" like is shown here. I always thought it was so elegant and had great potential, and I introduced a lot of people in my circle to it, but then it seems NVIDIA buried it. This blog post is great, and we need these kind of tools for sure, but we also…

Oh that sounds interesting. Do you know what happened to it? I think I found it here: https://github.com/bryancatanzaro/copperhead But I'm not sure what the state is. Looks dead (last commit 8 years ago). Probably just a proof of concept. But why hasn't this been continued? Blog post and example: https://developer.nvidia.com/blog/copperhead-data-parallel-p... https://github.com/bryancatanzaro/copperhead/blob/master/s…

I don't know specifics about Copperhead in particular, but Bryan Catanzaro (creator of Copperhead) is now the VP of Applied Deep Learning Research at Nvidia. He gave a talk at GTC this year, which is how I heard about all of this in the first place.

Source: https://www.linkedin.com/in/bryancatanzaro/

Re: Unifying the CUDA Python Ecosystem

#23

Post by klmadfejno » Fri, Apr 16, 2021, 5:28 PM UTC

jjoonathan wrote: post #21

andrew_v4 wrote: post #7

Just for contrast its interesting to look at an example of writing a similar kernel in Julia: https://juliagpu.gitlab.io/CUDA.jl/tutorials/introduction/ I don't think it's possible to achieve something like this in python because of how it's interpreted (but it sounds a bit like what another comment mentioned where the python was compiled to C)

> Julia has first-class support for GPU programming "First-class" is a steep claim. Does it support the nvidia perf tools? Those are very important for taking a kernel from (in my experience) ~20% theoretical perf to ~90% theoretical perf.

https://developer.nvidia.com/blog/gpu-computing-julia-progra...

Re: Unifying the CUDA Python Ecosystem

#24

Post by machineko » Fri, Apr 16, 2021, 5:31 PM UTC

andrew_v4 wrote: post #7

Just for contrast its interesting to look at an example of writing a similar kernel in Julia: https://juliagpu.gitlab.io/CUDA.jl/tutorials/introduction/ I don't think it's possible to achieve something like this in python because of how it's interpreted (but it sounds a bit like what another comment mentioned where the python was compiled to C)

Every time there is a topic about python, there is this one Julia guy who spam Julia "alternative" for python solution in every topic. Can you just guys stop? it kinda feels like watching a cult.

Re: Unifying the CUDA Python Ecosystem

#25

Post by jjoonathan » Fri, Apr 16, 2021, 5:32 PM UTC

rrss wrote: post #12

andrew_v4 wrote: post #7

Just for contrast its interesting to look at an example of writing a similar kernel in Julia: https://juliagpu.gitlab.io/CUDA.jl/tutorials/introduction/ I don't think it's possible to achieve something like this in python because of how it's interpreted (but it sounds a bit like what another comment mentioned where the python was compiled to C)

I think the contrast is probably less about the language, and more about the scope and objective of the projects. the blog is describing low-level interfaces in python - probably more comparable is the old CUDAdrv.jl package (now merged into CUDA.jl): https://github.com/JuliaGPU/CUDAdrv.jl/blob/master/examples/... here is writing a similar kernel in python with numba: https://github.com/ContinuumIO/gtc2017-numba/blob…

I gave numba CUDA a spin in late 2018 and was severely disappointed. It didn't work out of the box, I had to tweak the source to remove a reference to an API that had been removed from CUDA more than a year prior (and deprecated long ago). Then I ran into a bug when converting a float array to a double array -- I had to declare the types three different times and it still did a naive byte-copy rather than a conversion. Thanks to a background in numerics, the symptoms were obvious, but yikes. The problem that finally did us in was an inability to get buffers to correctly pass between kernels without a CPU copy, which was absolutely critical for our perf. I think this was supported in theory but just didn't work.

In any case, we did a complete rewrite in CUDA proper in less time than we spent banging our heads against that last numba-CUDA issue.

Under every language bridge there are trolls and numba-CUDA had some mean ones. Hopefully things have gotten better but I'm definitely still inside the "once bitten twice shy" period.

Re: Unifying the CUDA Python Ecosystem

#26

Post by jjoonathan » Fri, Apr 16, 2021, 5:40 PM UTC

klmadfejno wrote: post #23

jjoonathan wrote: post #21

Earlier quoted context omitted.

> Julia has first-class support for GPU programming "First-class" is a steep claim. Does it support the nvidia perf tools? Those are very important for taking a kernel from (in my experience) ~20% theoretical perf to ~90% theoretical perf.

https://developer.nvidia.com/blog/gpu-computing-julia-progra...

> CUDAnative.jl also [...] generates the necessary line number information for the NVIDIA Visual Profiler to work as expected

That sounds very promising, but these tools are usually magnificent screenshot fodder yet they are conspicuously absent from the screenshots so I still have suspicions. Maybe I'll give it a try tonight and report back.

Re: Unifying the CUDA Python Ecosystem

#27

Post by pjmlp » Fri, Apr 16, 2021, 5:45 PM UTC

nevi-me wrote: post #11

I have a RTX 2070 that's under-utilised, partly because I'm surprisingly finding it hard to understand C, C++ and CUDA by extension. I'm self-taught, and have been using web languages and some python, before learning Rust. I hope that NVIDIA can dedicate some resources to creating high-quality bindings to the C API for Rust, even if in the next 1-2 years. Perhaps being able to use a systems language that's been easy…

A nice thing of the proper ALGOL linage systems programming languages (which C only has basic influence), is that you can write nice high level code and only deal with pointers and raw pointer stuff when actually needed, think Ada, Modula-2, Object Pascal kind of languages.

So something like CUDA Rust would be nice to have.

By the way, D already supports CUDA,

https://dlang.org/blog/2017/07/17/dcompute-gpgpu-with-native...

Re: Unifying the CUDA Python Ecosystem

#28

Post by eslaught » Fri, Apr 16, 2021, 5:45 PM UTC

michelpp wrote: post #2

About 8 years ago an NVIDIA developer released a tool called Copperhead that let you write CUDA kernels in straight Python that were then compiled to C, no "C-in-a-string" like is shown here. I always thought it was so elegant and had great potential, and I introduced a lot of people in my circle to it, but then it seems NVIDIA buried it. This blog post is great, and we need these kind of tools for sure, but we also…

As it turns out, NVIDIA just open sourced a product called Legate which does not just GPUs but distributed as well. Right now it supports NumPy and Pandas but perhaps they'll add others in the future. Just thought this might be up your alley since it works at a higher level than the glorified CUDA in the article.

https://github.com/nv-legate/legate.numpy

Disclaimer: I work on the project they used to do the distributed execution, but otherwise have no connection with Legate.

Edit: And this library was developed by a team managed by one of the original Copperhead developers, in case you're wondering.

Re: Unifying the CUDA Python Ecosystem

#29

Post by sdajk3n423 » Fri, Apr 16, 2021, 5:49 PM UTC

nevi-me wrote: post #17

sdajk3n423 wrote: post #14

Earlier quoted context omitted.

If you are a looking to maximize use of that card, you can make about $5 a day mining crypto with the 2070.

No, the high electricity cost in my country + the noise pollution in the house + how much I generally earn from the machine + my views on burning the world speculatively, discourage me from mining crypto. Perhaps my position might change in future, but for now, I'd probably rather make the GPU accessible to those open-source distributed grids that train chess engines or compute deep-space related thingies :)

I am not convinced that training AI to win at chess is any more moral than mining crypto. And the block chain is about as open-source as you can get.

Re: Unifying the CUDA Python Ecosystem

#30

Post by maleadt » Fri, Apr 16, 2021, 5:50 PM UTC

jjoonathan wrote: post #21

andrew_v4 wrote: post #7

Just for contrast its interesting to look at an example of writing a similar kernel in Julia: https://juliagpu.gitlab.io/CUDA.jl/tutorials/introduction/ I don't think it's possible to achieve something like this in python because of how it's interpreted (but it sounds a bit like what another comment mentioned where the python was compiled to C)

> Julia has first-class support for GPU programming "First-class" is a steep claim. Does it support the nvidia perf tools? Those are very important for taking a kernel from (in my experience) ~20% theoretical perf to ~90% theoretical perf.

Yeah, see this section of the documentation: https://juliagpu.gitlab.io/CUDA.jl/development/profiling/. CUDA.jl also supports NVTX, wraps CUPTI, etc. The full extent of the APIs and tools is available.

Source line association when using PC sampling is currently broken due to a bug in the NVIDIA drivers though (segfaulting when parsing the PTX debug info emitted by LLVM), but I'm told that may be fixed in the next driver.