Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Unifying the CUDA Python Ecosystem

developer.nvidia.com

Post reply on HN
Print view View original

31–40 of 62 posts

Re: Unifying the CUDA Python Ecosystem

#31

Post by eslaught »

nuisance-bear wrote: post #4

Tools to make GPU development easier are sorely needed. I foolishly built an options pricing engine on top of PyTorch, thinking "oooh, it's a fast array library that supports CUDA transparently". Only to find out that array indexing is 100x slower than numpy.

You might be interested in Legate [1]. It supports the NumPy interface as a drop-in replacement, supports GPUs and also distributed machines. And you can see for yourself their performance results; they're not far off from hand-tuned MPI.

[1]: https://github.com/nv-legate/legate.numpy

Disclaimer: I work on the library Legate uses for distributed computing, but otherwise have no connection.

Re: Unifying the CUDA Python Ecosystem

#32

Post by maleadt »

jjoonathan wrote: post #26
klmadfejno wrote: post #23

Earlier quoted context omitted.

https://developer.nvidia.com/blog/gpu-computing-julia-progra...

> CUDAnative.jl also [...] generates the necessary line number information for the NVIDIA Visual Profiler to work as expected That sounds very promising, but these tools are usually magnificent screenshot fodder yet they are conspicuously absent from the screenshots so I still have suspicions. Maybe I'll give it a try tonight and report back.

Here's a screenshot: https://julialang.org/assets/blog/nvvp.png. Or a recent PR when you can see NVTX ranges from Julia: https://github.com/JuliaGPU/CUDA.jl/pull/760

Re: Unifying the CUDA Python Ecosystem

#33

Post by TuringNYC »

nuisance-bear wrote: post #4

Tools to make GPU development easier are sorely needed. I foolishly built an options pricing engine on top of PyTorch, thinking "oooh, it's a fast array library that supports CUDA transparently". Only to find out that array indexing is 100x slower than numpy.

>>> built an options pricing engine on top of PyTorch

I'd love to hear more about this! Do you have any posts or write-ups on this?

Re: Unifying the CUDA Python Ecosystem

#34

Post by jjoonathan »

maleadt wrote: post #30
jjoonathan wrote: post #21

Earlier quoted context omitted.

> Julia has first-class support for GPU programming "First-class" is a steep claim. Does it support the nvidia perf tools? Those are very important for taking a kernel from (in my experience) ~20% theoretical perf to ~90% theoretical perf.

Yeah, see this section of the documentation: https://juliagpu.gitlab.io/CUDA.jl/development/profiling/ . CUDA.jl also supports NVTX, wraps CUPTI, etc. The full extent of the APIs and tools is available. Source line association when using PC sampling is currently broken due to a bug in the NVIDIA drivers though (segfaulting when parsing the PTX debug info emitted by LLVM), but I'm told that may be fixed in the next dr…

Nice! I set a reminder to check back in a month.

Re: Unifying the CUDA Python Ecosystem

#35

Post by jjoonathan »

maleadt wrote: post #32
jjoonathan wrote: post #26

Earlier quoted context omitted.

> CUDAnative.jl also [...] generates the necessary line number information for the NVIDIA Visual Profiler to work as expected That sounds very promising, but these tools are usually magnificent screenshot fodder yet they are conspicuously absent from the screenshots so I still have suspicions. Maybe I'll give it a try tonight and report back.

Here's a screenshot: https://julialang.org/assets/blog/nvvp.png . Or a recent PR when you can see NVTX ranges from Julia: https://github.com/JuliaGPU/CUDA.jl/pull/760

Thanks! Now I believe! :)

Re: Unifying the CUDA Python Ecosystem

#36

Post by kolbe »

As someone who has dabbled in CUDA with some success, I'm going to be a little contrarian here. To me, the difficulty with GPU programming isn't the fact that CUDA uses C-syntax versus something more readable like Python. GPU programming is fundamentally difficult, and the minor gains from using a familiar language syntax are dwarfed by the need to understand blocks, memory alignment, thread hierarchy, etc. And I don't just say this. I live it. Even though I primarily program in C#, I don't use Hybridizer when I need GPU acceleration. I go straight to CUDA and marshal everything to/from C#.

That's not to say that CUDA Python isn't kinda cool, but it's not a magic bullet to finally understanding GPU programming if you've been struggling.

Re: Unifying the CUDA Python Ecosystem

#37

Post by [deleted] »

eslaught wrote: post #31
nuisance-bear wrote: post #4

Tools to make GPU development easier are sorely needed. I foolishly built an options pricing engine on top of PyTorch, thinking "oooh, it's a fast array library that supports CUDA transparently". Only to find out that array indexing is 100x slower than numpy.

You might be interested in Legate [1]. It supports the NumPy interface as a drop-in replacement, supports GPUs and also distributed machines. And you can see for yourself their performance results; they're not far off from hand-tuned MPI. [1]: https://github.com/nv-legate/legate.numpy Disclaimer: I work on the library Legate uses for distributed computing, but otherwise have no connection.

[deleted]

Re: Unifying the CUDA Python Ecosystem

#38

Post by sideshowb »

nuisance-bear wrote: post #4

Tools to make GPU development easier are sorely needed. I foolishly built an options pricing engine on top of PyTorch, thinking "oooh, it's a fast array library that supports CUDA transparently". Only to find out that array indexing is 100x slower than numpy.

Interesting find about the indexing. I just had the opposite experience, swapped from numpy to torch in a project and got 2000x speedup on some indexing and basic maths wrapped in autodiff. And I haven't moved it onto cuda yet.

Re: Unifying the CUDA Python Ecosystem

#39

Post by touisteur »

pjmlp wrote: post #27
nevi-me wrote: post #11

I have a RTX 2070 that's under-utilised, partly because I'm surprisingly finding it hard to understand C, C++ and CUDA by extension. I'm self-taught, and have been using web languages and some python, before learning Rust. I hope that NVIDIA can dedicate some resources to creating high-quality bindings to the C API for Rust, even if in the next 1-2 years. Perhaps being able to use a systems language that's been easy…

A nice thing of the proper ALGOL linage systems programming languages (which C only has basic influence), is that you can write nice high level code and only deal with pointers and raw pointer stuff when actually needed, think Ada, Modula-2, Object Pascal kind of languages. So something like CUDA Rust would be nice to have. By the way, D already supports CUDA, https://dlang.org/blog/2017/07/17/dcompute-gpgpu-with-nat…

CUDA Ada would be so, so nice. Especially with non-aliasing guarantees from SPARK...

Re: Unifying the CUDA Python Ecosystem

#40

Post by nuisance-bear »

sideshowb wrote: post #38
nuisance-bear wrote: post #4

Tools to make GPU development easier are sorely needed. I foolishly built an options pricing engine on top of PyTorch, thinking "oooh, it's a fast array library that supports CUDA transparently". Only to find out that array indexing is 100x slower than numpy.

Interesting find about the indexing. I just had the opposite experience, swapped from numpy to torch in a project and got 2000x speedup on some indexing and basic maths wrapped in autodiff. And I haven't moved it onto cuda yet.

Here's an example that illustrates the phenomenon. If memory serves me right, index latency is superlinear in dimension count.

   import time, torch
   from itertools import product

   N = 100

   ten = torch.randn(N,N,N)
   arr = ten.numpy()

   def indexTimer(val):
       start = time.time()
       for i,j,k in product(range(N), range(N), range(N)):
           x = val[i, j, k]
       end = time.time()
       print('{:.2f}'.format(end-start))

   indexTimer(ten)
   indexTimer(arr)
Post reply on HN