Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Aiter: AI Tensor Engine for ROCm

rocm.blogs.amd.com

Post reply on HN
Print view View original

31–40 of 92 posts

Re: Aiter: AI Tensor Engine for ROCm

#31

Post by shihab »

fancyfredbot wrote: post #20

Wow, it sure sounds like a mess under there. They used 4 different languages? Using one high level language and assembly sounds fine, but four feels incoherent. Would love to know why this has had happened. "This infrastructure is built upon a variety of underlying technologies, including Triton, CK (Compute Kernel), ASM (Assembly), and HIP (Heterogeneous Interface for Portability)."

Wait, did they get their own library name wrong? CK should be Composable Kernel, I can’t find anything called compute kernel anywhere

Re: Aiter: AI Tensor Engine for ROCm

#32

Post by ColonelPhantom »

fancyfredbot wrote: post #23
jfim wrote: post #22

Earlier quoted context omitted.

That's not exactly unusual, for example pytorch has Python, C++, C, and Cuda.

Notice those are all (except arguably CUDA) very mainstream languages. All four of AMDs are niche. Upstreaming this into pytorch would double the number of languages used. (Although HIP is very similar to CUDA)

I believe that PyTorch already uses Triton; I recently tried to do torch.compile on a Windows machine and it did not work because the inductor backend relies on Triton.

Re: Aiter: AI Tensor Engine for ROCm

#35

Post by almostgotcaught »

imtringued wrote: post #19
fock wrote: post #9

Earlier quoted context omitted.

I think they are taken over by exactly the same people leading the AI-hype. Funny how in this article they are a) not advertising clearly what they are doing, b) solving a small subset of problems in a way noone asked for (I think most people just want ROCm to work at all...) and c) just adding to a complex product without any consideration of actually integrating with its environment. I guess it's vibecoding "AI"...

They are imitating Nvidia's TensorRT with AITER. Basically AMD wants to have "CUDA, but not CUDA".

> They are imitating Nvidia's TensorRT

Do you know what the RT in TensorRT stands for? hint: AITER has nothing to do with TensorRT.

Re: Aiter: AI Tensor Engine for ROCm

#36

Post by fancyfredbot »

shihab wrote: post #31
fancyfredbot wrote: post #20

Wow, it sure sounds like a mess under there. They used 4 different languages? Using one high level language and assembly sounds fine, but four feels incoherent. Would love to know why this has had happened. "This infrastructure is built upon a variety of underlying technologies, including Triton, CK (Compute Kernel), ASM (Assembly), and HIP (Heterogeneous Interface for Portability)."

Wait, did they get their own library name wrong? CK should be Composable Kernel, I can’t find anything called compute kernel anywhere

It does look like that yes. It wasn't my error, the quote is copy pasted verbatim from the article.

Re: Aiter: AI Tensor Engine for ROCm

#37

Post by mjburgess »

Any one try any of this on a few 7900xtx (or familiarity with this hardware and platform)? I've just purchased 6 for some small-scale experimentation. I'm thinking the next machine I'll use AMD Radeon PRO W7900 (to get 128 GB VRAM / machine).

Re: Aiter: AI Tensor Engine for ROCm

#38

Post by [deleted] »

evertedsphere wrote: post #3
carbocation wrote: post #2

I'm not a python expert, but this feels very odd to me (both the * init* construction and the return [tgemm.mm]( http://tgemm.mm/ )(input, self.weight, self.bias, None, None) call, which looks like markdown to me: from aiter.tuned_gemm import tgemm import torch class LinearLayer(torch.nn.Module): def **init**(self, in_features, out_features): super(LinearLayer, self).**init**() self.weight = torch.nn.Parameter(torch.…

yep the syntax highlighting / doc hyperlinking clearly broke there (or, less charitably, whatever llm produced that prose had a moment) it's __init__ of course

[deleted]

Re: Aiter: AI Tensor Engine for ROCm

#39

Post by almostgotcaught »

mjburgess wrote: post #37

Any one try any of this on a few 7900xtx (or familiarity with this hardware and platform)? I've just purchased 6 for some small-scale experimentation. I'm thinking the next machine I'll use AMD Radeon PRO W7900 (to get 128 GB VRAM / machine).

Just export HSA_OVERRIDE_GFX_VERSION=11.0.0 and things should mostly work. Off the top of my head, some of the fp8 types aren't supported but

Re: Aiter: AI Tensor Engine for ROCm

#40

Post by TheCycoONE »

ColonelPhantom wrote: post #30
Rounin wrote: post #16

Earlier quoted context omitted.

You should be able to make it think you have another card: export HSA_OVERRIDE_GFX_VERSION=10.3.0 The possible values are said to be: # gfx1030 = "10.3.0" # gfx900 = "9.0.0" # gfx906 = "9.0.6" # gfx908 = "9.0.8" # gfx90a = "9.0.a"

I believe the override for GP's 7600 is 1100 or 11.0.0 as GFX1030 is RDNA2 (6800 XT).

The 7900 models are all 1100, the 7800XT is 1101 and the 7600 is 1102.

See Shader ISA: https://www.techpowerup.com/gpu-specs/radeon-rx-7600-xt.c419...

Post reply on HN