Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Triton: Open-Source GPU Programming for Neural Networks

openai.com

Post reply on HN
Print view View original

61–70 of 116 posts

Re: Triton: Open-Source GPU Programming for Neural Networks

#61

Post by enos_feedler »

dragontamer wrote: post #20
zozbot234 wrote: post #18

Earlier quoted context omitted.

The new contest is not CUDA vs. OpenCL but CUDA vs. Vulkan Compute. As support for Vulkan in hardware becomes more widespread, it makes more and more sense to just standardize on it for all workloads. The programming model is quite different between the two (kernels vs. shaders) and OpenCL 2.x has quite a few features that are not in Vulkan, but the latest version of OpenCL has downgraded many of these to extensions.

CUDA programmers choose CUDA because when you make a struct FooBar{}; in CUDA, it works on both CPU-side and GPU-side. Vulkan / OpenCL / etc. etc. don't have any data-structure sharing like that with the host code. Its a point of contention that makes anything more complicated than a 3-dimensional array hard to share. Yeah, Vulkan / OpenCL have all sorts of pointer-sharing arrangements (Shared Virtual Memory) or what…

Is it practical for code to flow between these two execution units? In my understanding, the architecture of a GPU and CPU are so fundamentally different that it doesn’t reallly make sense. You do want to have a single data structure representation however.

Re: Triton: Open-Source GPU Programming for Neural Networks

#62

Post by einpoklum »

dragontamer wrote: post #44
einpoklum wrote: post #41

Earlier quoted context omitted.

> ROCm from AMD has its restrictions, but... it really is easier to program than OpenCL That really not the point. OpenCL is a standard that - at least in principle - is supposed to be supported on multiple platforms by multiple vendors. ROCm is AMD-only, and even that is questionable since it didn't exist 6 or 7 years ago, and who knows - they might drop it like they changed their earlier focus. Also, CUDA has a muc…

> OpenCL is a standard that - at least in principle - is supposed to be supported on multiple platforms by multiple vendors. As was C++AMP (which was actually pretty good IMO as a language). Just because its a standard doesn't mean its going to be used. OpenCL 2.0 was very poorly implemented: almost no one used any of its advanced features. To the point that OpenCL 3.0 is resetting from OpenCL 1.2. Only Intel really…

Pretty much agree; and I'll add NVIDIA has also made it a point not to support profiling (and probably debugging) OpenCL... to this day.

Still, that doesn't contradict what I said in my earlier post.

There was also OpenCL C++, which I really wanted to see implemented by GPU vendors, especially NVIDIA, and that didn't happen either.

As for C++AMP - did that ever work on NVIDIA hardware?

Re: Triton: Open-Source GPU Programming for Neural Networks

#63

Post by dragontamer »

enos_feedler wrote: post #61
dragontamer wrote: post #20

Earlier quoted context omitted.

CUDA programmers choose CUDA because when you make a struct FooBar{}; in CUDA, it works on both CPU-side and GPU-side. Vulkan / OpenCL / etc. etc. don't have any data-structure sharing like that with the host code. Its a point of contention that makes anything more complicated than a 3-dimensional array hard to share. Yeah, Vulkan / OpenCL have all sorts of pointer-sharing arrangements (Shared Virtual Memory) or what…

Is it practical for code to flow between these two execution units? In my understanding, the architecture of a GPU and CPU are so fundamentally different that it doesn’t reallly make sense. You do want to have a single data structure representation however.

> In my understanding, the architecture of a GPU and CPU are so fundamentally different that it doesn’t reallly make sense.

Yes and no. You find surprising bits of code that can be shared.

Read through a high-performance GPU project like "GPU perft" (https://github.com/ankan-ban/perft_gpu/blob/master/chess.h). The "perft" problem is the problem of counting the number of board positions reachable in X moves. (Perft(15) is "how many board positions exist within 15 moves from the start of chess?". Perft(1) is 20, and Perft(2) is 400, then 3+ gets a bit complicated!).

You can see that the fundamentals of bitboard manipulation (using the 64-bit number to represent the 64-squares of a chessboard) remain the same whether or not you're on a CPU or GPU.

In this case, you want the code to be shared between the two sides. Why write the code twice? Both CPUs and GPUs are very good at 64-bit integer manipulation.

------------

The actual search (how to branch off, coalesce results, coordinate threads) is extremely different between CPU and GPU. So of course, you want that code to be written in a CPU-specific, or GPU-specific manner.

But the question of "where can this Bishop move??" (the sliding piece attack subroutine) is identical on CPU or GPU.

Re: Triton: Open-Source GPU Programming for Neural Networks

#64

Post by etaioinshrdlu »

37ef_ced3 wrote: post #36
shubuZ wrote: post #9

I have found writing CUDA code is much simpler than writing correct multi-threaded AVX2/AVX-512 code.

Use a domain-specific compiler to generate custom, stand-alone, massively multi-threaded AVX-512 inference C code: https://NN-512.com The generated code is easily twice as fast as TensorFlow's AVX-512 kernels (Intel's oneAPI).

Your project is well engineered but no matter how many times you post it here, you won’t get real traction without a different development model, such as building on github. This is especially true in a high churn, high cost field like ML. I also think you are being anonymous unnecessarily.

Re: Triton: Open-Source GPU Programming for Neural Networks

#66

Post by blt »

croes wrote: post #4

Too bad it's CUDA Sooner or later this will become a problem because you are depending on the benevolence of a single manufacturer.

Forgive my ignorance of legal matters, but could AMD write a CUDA compiler now or in the future? After some patent expires?

Re: Triton: Open-Source GPU Programming for Neural Networks

#67

Post by enos_feedler »

dragontamer wrote: post #63
enos_feedler wrote: post #61

Earlier quoted context omitted.

Is it practical for code to flow between these two execution units? In my understanding, the architecture of a GPU and CPU are so fundamentally different that it doesn’t reallly make sense. You do want to have a single data structure representation however.

> In my understanding, the architecture of a GPU and CPU are so fundamentally different that it doesn’t reallly make sense. Yes and no. You find surprising bits of code that can be shared. Read through a high-performance GPU project like "GPU perft" ( https://github.com/ankan-ban/perft_gpu/blob/master/chess.h ). The "perft" problem is the problem of counting the number of board positions reachable in X moves. (Perft(…

Sounds similar to business logic shared between and client and server web app. Thanks

I just wonder if this alone is enough to make a unified programming environment the preferred way. On the web we had Meteor.js which tried this approach to unify client and server with javascript. The shared code was of similar types. Meteor never became the preferred way to write web apps. Im wondering if the same is true for GPU + CPU programming.

Re: Triton: Open-Source GPU Programming for Neural Networks

#69

Post by dragontamer »

einpoklum wrote: post #62
dragontamer wrote: post #44

Earlier quoted context omitted.

> OpenCL is a standard that - at least in principle - is supposed to be supported on multiple platforms by multiple vendors. As was C++AMP (which was actually pretty good IMO as a language). Just because its a standard doesn't mean its going to be used. OpenCL 2.0 was very poorly implemented: almost no one used any of its advanced features. To the point that OpenCL 3.0 is resetting from OpenCL 1.2. Only Intel really…

Pretty much agree; and I'll add NVIDIA has also made it a point not to support profiling (and probably debugging) OpenCL... to this day. Still, that doesn't contradict what I said in my earlier post. There was also OpenCL C++, which I really wanted to see implemented by GPU vendors, especially NVIDIA, and that didn't happen either. As for C++AMP - did that ever work on NVIDIA hardware?

C++ AMP compiled into Microsoft's DirectCompute which works on Nvidia (through the video game DirectX stuff)

Re: Triton: Open-Source GPU Programming for Neural Networks

#70

Post by N1H1L »

riyadparvez wrote: post #2

I am confused. Is it another competitor of Tensorflow, JAX, and Pytorch? Or something else?

From what I got from reading the docs and the blog post, was that this is a competitor to torch.jit and numba.cuda.jit - write faster Pythonic GPGPU kernels without sacrificing speed.
Post reply on HN