Silly question perhaps, but is this a true CUDA equivalent? Why (not)?
Aiter: AI Tensor Engine for ROCm
21–30 of 92 posts
Re: Aiter: AI Tensor Engine for ROCm
#22Wow, it sure sounds like a mess under there. They used 4 different languages? Using one high level language and assembly sounds fine, but four feels incoherent. Would love to know why this has had happened. "This infrastructure is built upon a variety of underlying technologies, including Triton, CK (Compute Kernel), ASM (Assembly), and HIP (Heterogeneous Interface for Portability)."
That's not exactly unusual, for example pytorch has Python, C++, C, and Cuda.
Re: Aiter: AI Tensor Engine for ROCm
#23Wow, it sure sounds like a mess under there. They used 4 different languages? Using one high level language and assembly sounds fine, but four feels incoherent. Would love to know why this has had happened. "This infrastructure is built upon a variety of underlying technologies, including Triton, CK (Compute Kernel), ASM (Assembly), and HIP (Heterogeneous Interface for Portability)."
That's not exactly unusual, for example pytorch has Python, C++, C, and Cuda.
Notice those are all (except arguably CUDA) very mainstream languages. All four of AMDs are niche. Upstreaming this into pytorch would double the number of languages used. (Although HIP is very similar to CUDA)
Re: Aiter: AI Tensor Engine for ROCm
#24I'm not a python expert, but this feels very odd to me (both the * init* construction and the return [tgemm.mm]( http://tgemm.mm/ )(input, self.weight, self.bias, None, None) call, which looks like markdown to me: from aiter.tuned_gemm import tgemm import torch class LinearLayer(torch.nn.Module): def **init**(self, in_features, out_features): super(LinearLayer, self).**init**() self.weight = torch.nn.Parameter(torch.…
I was puzzling over the code wondering why they .cuda() everything like that when I realised that that was only the beginning of the weirdness.
I'm assuming the scrambled annotations were due to some odd chain of things the code went through on the way to becoming a post.
Maybe they did it as a parable about the problems of having many layers of abstraction causing processes with unintended consequences?
Re: Aiter: AI Tensor Engine for ROCm
#25Wow, it sure sounds like a mess under there. They used 4 different languages? Using one high level language and assembly sounds fine, but four feels incoherent. Would love to know why this has had happened. "This infrastructure is built upon a variety of underlying technologies, including Triton, CK (Compute Kernel), ASM (Assembly), and HIP (Heterogeneous Interface for Portability)."
Those aren't four different languages. CK and HIP are both just libraries.
Re: Aiter: AI Tensor Engine for ROCm
#26Earlier quoted context omitted.
I think they are taken over by exactly the same people leading the AI-hype. Funny how in this article they are a) not advertising clearly what they are doing, b) solving a small subset of problems in a way noone asked for (I think most people just want ROCm to work at all...) and c) just adding to a complex product without any consideration of actually integrating with its environment. I guess it's vibecoding "AI"...
solving a small subset of problems in a way noone asked for What do you mean? Having ROCm fused MoE and MLA kernels as a counterpart to kernels for CUDA is very useful. AMD needs to provide this if they want to keep AMD accelerators competitive with new models.
should the matrix-multiplication at the core of this not be in a core library? Why are generic layers intermixed with LLM-specific kernels when the generic layers are duplicating functionality in torch?
Upstreaming that might actually help researchers doing new stuff vs. the narrow demographic of people speeding LLMs on MI300X's.
Re: Aiter: AI Tensor Engine for ROCm
#27Earlier quoted context omitted.
I think they are taken over by exactly the same people leading the AI-hype. Funny how in this article they are a) not advertising clearly what they are doing, b) solving a small subset of problems in a way noone asked for (I think most people just want ROCm to work at all...) and c) just adding to a complex product without any consideration of actually integrating with its environment. I guess it's vibecoding "AI"...
They are imitating Nvidia's TensorRT with AITER. Basically AMD wants to have "CUDA, but not CUDA".
They'd like to have CUDA, period, but are legally barred from it.
Re: Aiter: AI Tensor Engine for ROCm
#28I just want to remind everyone that El Capitan, Frontier and LUMI supercomputers are powered by AMD instinct cards.
El Capitan is #1 in TOP500. Frontier is #2, LUMI is #8.
ROCm development is probably mainly driven by the needs of these supercompuers' users currently.
So, we're seeing the tip of the iceberg.
Also ROCm packages continue to land on Debian, so there's more than meets the eye.
Note: Search "AMD Instinct" at https://top500.org/lists/top500/list/2024/11/. There are way more systems.
Re: Aiter: AI Tensor Engine for ROCm
#29Wow, it sure sounds like a mess under there. They used 4 different languages? Using one high level language and assembly sounds fine, but four feels incoherent. Would love to know why this has had happened. "This infrastructure is built upon a variety of underlying technologies, including Triton, CK (Compute Kernel), ASM (Assembly), and HIP (Heterogeneous Interface for Portability)."
Those aren't four different languages. CK and HIP are both just libraries.
HIP is AMD's equivalent of CUDA and is certainly a language.
But you are right CK is indeed a library, thanks for pointing that out.
Re: Aiter: AI Tensor Engine for ROCm
#30Still waiting for ROCm on my cheap Radeon RX 7600. Would be nice to play around with it a little. I know that this card is nothing fancy. There is somewhere a github issue where they announced to port it for linux to consumer cards, but last time I checked (a few days ago) it still wasn't available
You should be able to make it think you have another card: export HSA_OVERRIDE_GFX_VERSION=10.3.0 The possible values are said to be: # gfx1030 = "10.3.0" # gfx900 = "9.0.0" # gfx906 = "9.0.6" # gfx908 = "9.0.8" # gfx90a = "9.0.a"
I believe the override for GP's 7600 is 1100 or 11.0.0 as GFX1030 is RDNA2 (6800 XT).