Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Aiter: AI Tensor Engine for ROCm

rocm.blogs.amd.com

Post reply on HN
Print view View original

1–10 of 92 posts

Re: Aiter: AI Tensor Engine for ROCm

#2

Post by carbocation »

I'm not a python expert, but this feels very odd to me (both the *init* construction and the return [tgemm.mm](http://tgemm.mm/)(input, self.weight, self.bias, None, None) call, which looks like markdown to me:

    from aiter.tuned_gemm import tgemm
    import torch
    
    class LinearLayer(torch.nn.Module):
     def **init**(self, in_features, out_features):
      super(LinearLayer, self).**init**()
      self.weight = torch.nn.Parameter(torch.randn(out_features, in_features).cuda())
      self.bias = torch.nn.Parameter(torch.randn(out_features).cuda())
    
     def forward(self, input):
      input = input.cuda()
      return [tgemm.mm](http://tgemm.mm/)(input, self.weight, self.bias, None, None)

Re: Aiter: AI Tensor Engine for ROCm

#3

Post by evertedsphere »

carbocation wrote: post #2

I'm not a python expert, but this feels very odd to me (both the * init* construction and the return [tgemm.mm]( http://tgemm.mm/ )(input, self.weight, self.bias, None, None) call, which looks like markdown to me: from aiter.tuned_gemm import tgemm import torch class LinearLayer(torch.nn.Module): def **init**(self, in_features, out_features): super(LinearLayer, self).**init**() self.weight = torch.nn.Parameter(torch.…

yep the syntax highlighting / doc hyperlinking clearly broke there (or, less charitably, whatever llm produced that prose had a moment)

it's __init__ of course

Re: Aiter: AI Tensor Engine for ROCm

#4

Post by Tpt »

If I understand correctly, this library provides some Torch kernels customized for AMD hardware. Why haven't they just upstreamed them to PyTorch for better adoption? Also, they seem to demo usage with Torch default eager execution mode and not Torch JIT/TorchScript. Is this library compatible with TorchScript?

Re: Aiter: AI Tensor Engine for ROCm

#5

Post by dailykoder »

Still waiting for ROCm on my cheap Radeon RX 7600. Would be nice to play around with it a little. I know that this card is nothing fancy. There is somewhere a github issue where they announced to port it for linux to consumer cards, but last time I checked (a few days ago) it still wasn't available

Re: Aiter: AI Tensor Engine for ROCm

#6

Post by barrenko »

Tpt wrote: post #4

If I understand correctly, this library provides some Torch kernels customized for AMD hardware. Why haven't they just upstreamed them to PyTorch for better adoption? Also, they seem to demo usage with Torch default eager execution mode and not Torch JIT/TorchScript. Is this library compatible with TorchScript?

I really do not understand why can't they just work with existing OSS developers pulling their hair out trying to make AMD devices work and instead do it this way. It's like Mozilla with the questionable decisions.

Re: Aiter: AI Tensor Engine for ROCm

#7

Post by fock »

carbocation wrote: post #2

I'm not a python expert, but this feels very odd to me (both the * init* construction and the return [tgemm.mm]( http://tgemm.mm/ )(input, self.weight, self.bias, None, None) call, which looks like markdown to me: from aiter.tuned_gemm import tgemm import torch class LinearLayer(torch.nn.Module): def **init**(self, in_features, out_features): super(LinearLayer, self).**init**() self.weight = torch.nn.Parameter(torch.…

also why is it calling .cuda() to move tensors to a cuda driver? I suppose this is because this is based on HIP - which comes with it's own set of problems, but that's ROCm for the masses I guess.

Also the tgemm.mm has to be a torch module (at first I thought this was some lowlevel library which they now have a preview of, because there is a ROCm-torch already ...) which is evident from the table just before the summary. That table also smells like they are mostly focused on inference...

EDIT: seems official ROCm-torch is also based on HIP.

Re: Aiter: AI Tensor Engine for ROCm

#8

Post by hoomanmo »

barrenko wrote: post #6
Tpt wrote: post #4

If I understand correctly, this library provides some Torch kernels customized for AMD hardware. Why haven't they just upstreamed them to PyTorch for better adoption? Also, they seem to demo usage with Torch default eager execution mode and not Torch JIT/TorchScript. Is this library compatible with TorchScript?

I really do not understand why can't they just work with existing OSS developers pulling their hair out trying to make AMD devices work and instead do it this way. It's like Mozilla with the questionable decisions.

which Mozilla's questionable decisions are you referring to?

Re: Aiter: AI Tensor Engine for ROCm

#9

Post by fock »

barrenko wrote: post #6
Tpt wrote: post #4

If I understand correctly, this library provides some Torch kernels customized for AMD hardware. Why haven't they just upstreamed them to PyTorch for better adoption? Also, they seem to demo usage with Torch default eager execution mode and not Torch JIT/TorchScript. Is this library compatible with TorchScript?

I really do not understand why can't they just work with existing OSS developers pulling their hair out trying to make AMD devices work and instead do it this way. It's like Mozilla with the questionable decisions.

I think they are taken over by exactly the same people leading the AI-hype. Funny how in this article they are a) not advertising clearly what they are doing, b) solving a small subset of problems in a way noone asked for (I think most people just want ROCm to work at all...) and c) just adding to a complex product without any consideration of actually integrating with its environment.

I guess it's vibecoding "AI"...

Re: Aiter: AI Tensor Engine for ROCm

#10

Post by oynqr »

dailykoder wrote: post #5

Still waiting for ROCm on my cheap Radeon RX 7600. Would be nice to play around with it a little. I know that this card is nothing fancy. There is somewhere a github issue where they announced to port it for linux to consumer cards, but last time I checked (a few days ago) it still wasn't available

I used rocm on an RX 7600 a month after launch. Having no official support does not at all mean it doesn't work.
Post reply on HN