Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Triton: Open-Source GPU Programming for Neural Networks

openai.com

Post reply on HN
Print view View original

11–20 of 116 posts

Re: Triton: Open-Source GPU Programming for Neural Networks

#11

Post by dragontamer »

croes wrote: post #4

Too bad it's CUDA Sooner or later this will become a problem because you are depending on the benevolence of a single manufacturer.

AMD's ROCm 4.0 now supports cooperative groups, which is probably one of the last major holdouts for CUDA compatibility.

There's still the 64-wavefront (for AMD CDNA cards) instead of 32-wavefronts (for CUDA). But AMD even has 4x4 half-float matrix multiplication instructions in ROCm (for MI100, the only card that supports the matrix-multiplication / tensor instructions)

---------

I think CUDA vs OpenCL is over. ROCm from AMD has its restrictions, but... it really is easier to program than OpenCL. Its a superior model: having a single language that supports both CPU and GPU code is just easier than switching between C++ and OpenCL (where data-structures can't be shared as easily).

-----------

The main issue with AMD is that they're cutting support for their older cards. The cheapest card you can get that supports ROCm is Vega56 now... otherwise you're basically expected to go for the expensive MI-line (MI50, MI100).

Re: Triton: Open-Source GPU Programming for Neural Networks

#12

Post by jpf0 »

blueblisters wrote: post #6
peytoncasper wrote: post #3

Earlier quoted context omitted.

I believe this is more of an optimization layer to be utilized by libraries like Tensorflow and JAX. More of a simplification of the interaction with traditional CUDA instructions. I imagine these libraries and possibly some users would implement libraries on top of this language and reap some of the optimization benefit without having to maintain low-level CUDA specific code.

So is this similar to XLA?

XLA is domain-specific compiler for linear algebra. Triton generates and compiles an intermediate representation for tiled computation. This IR allows more general functions and also claims higher performance.

obligatory reference to the family of work: https://github.com/merrymercy/awesome-tensor-compilers

Re: Triton: Open-Source GPU Programming for Neural Networks

#17

Post by polynomial »

thebruce87m wrote: post #14

Unfortunate name clash with NVIDIAs Triton Inference Server: https://developer.nvidia.com/nvidia-triton-inference-server

My first thought exactly. This will cause nothing but confusion and Triton (the inference server) is well integrated into the space.

So it's especially weird to see it coming from OpenAI, and not a more random startup. It honestly makes no sense they would deliberately do this, unless there is some secret cult of Triton that is going on in the Bay Area world of AI/ML.

Re: Triton: Open-Source GPU Programming for Neural Networks

#18

Post by zozbot234 »

dragontamer wrote: post #11
croes wrote: post #4

Too bad it's CUDA Sooner or later this will become a problem because you are depending on the benevolence of a single manufacturer.

AMD's ROCm 4.0 now supports cooperative groups, which is probably one of the last major holdouts for CUDA compatibility. There's still the 64-wavefront (for AMD CDNA cards) instead of 32-wavefronts (for CUDA). But AMD even has 4x4 half-float matrix multiplication instructions in ROCm (for MI100, the only card that supports the matrix-multiplication / tensor instructions) --------- I think CUDA vs OpenCL is over. ROCm…

The new contest is not CUDA vs. OpenCL but CUDA vs. Vulkan Compute. As support for Vulkan in hardware becomes more widespread, it makes more and more sense to just standardize on it for all workloads. The programming model is quite different between the two (kernels vs. shaders) and OpenCL 2.x has quite a few features that are not in Vulkan, but the latest version of OpenCL has downgraded many of these to extensions.

Re: Triton: Open-Source GPU Programming for Neural Networks

#19

Post by dragontamer »

shubuZ wrote: post #9

I have found writing CUDA code is much simpler than writing correct multi-threaded AVX2/AVX-512 code.

If you need CPU-side SIMD, then try ispc: https://ispc.github.io/

Its pretty much the OpenCL-model, except it compiles into AVX2 code / AVX512 code. Very similar to CUDA / OpenCL style programming. Its not single-source like CUDA, but it largely accomplishes the programming model IMO.

Re: Triton: Open-Source GPU Programming for Neural Networks

#20

Post by dragontamer »

zozbot234 wrote: post #18
dragontamer wrote: post #11

Earlier quoted context omitted.

AMD's ROCm 4.0 now supports cooperative groups, which is probably one of the last major holdouts for CUDA compatibility. There's still the 64-wavefront (for AMD CDNA cards) instead of 32-wavefronts (for CUDA). But AMD even has 4x4 half-float matrix multiplication instructions in ROCm (for MI100, the only card that supports the matrix-multiplication / tensor instructions) --------- I think CUDA vs OpenCL is over. ROCm…

The new contest is not CUDA vs. OpenCL but CUDA vs. Vulkan Compute. As support for Vulkan in hardware becomes more widespread, it makes more and more sense to just standardize on it for all workloads. The programming model is quite different between the two (kernels vs. shaders) and OpenCL 2.x has quite a few features that are not in Vulkan, but the latest version of OpenCL has downgraded many of these to extensions.

CUDA programmers choose CUDA because when you make a struct FooBar{}; in CUDA, it works on both CPU-side and GPU-side.

Vulkan / OpenCL / etc. etc. don't have any data-structure sharing like that with the host code. Its a point of contention that makes anything more complicated than a 3-dimensional array hard to share.

Yeah, Vulkan / OpenCL have all sorts of pointer-sharing arrangements (Shared Virtual Memory) or whatnot. But its difficult to use in practice, because they keep the concepts of "GPU" code separate from "CPU" code.

---------

When you look at these things: such as Triton, you see that people want to unify the CPU-and-GPU code into a single code base. Look and read these Triton examples: they're just Python code, inside of the rest of CPU-Python code.

I think people are realizing that high-level code can flow between the two execution units (CPU or GPU) without changing the high level language. The compiler works hard to generate code for both systems, but its better for the compiler to work rather than the programmer to work on integration.

Post reply on HN