Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Aiter: AI Tensor Engine for ROCm

rocm.blogs.amd.com

Post reply on HN
Print view View original

61–70 of 92 posts

Re: Aiter: AI Tensor Engine for ROCm

#61

Post by saagarjha »

bayindirh wrote: post #28

I just want to remind everyone that El Capitan, Frontier and LUMI supercomputers are powered by AMD instinct cards. El Capitan is #1 in TOP500. Frontier is #2, LUMI is #8. ROCm development is probably mainly driven by the needs of these supercompuers' users currently. So, we're seeing the tip of the iceberg. Also ROCm packages continue to land on Debian, so there's more than meets the eye. Note: Search "AMD Instinct"…

> ROCm development is probably mainly driven by the needs of these supercompuers' users currently.

Seems like a problem since AMD wants to go after AI capex?

Re: Aiter: AI Tensor Engine for ROCm

#62

Post by saagarjha »

kouteiheika wrote: post #17
Tpt wrote: post #4

If I understand correctly, this library provides some Torch kernels customized for AMD hardware. Why haven't they just upstreamed them to PyTorch for better adoption? Also, they seem to demo usage with Torch default eager execution mode and not Torch JIT/TorchScript. Is this library compatible with TorchScript?

> Why haven't they just upstreamed them to PyTorch for better adoption? They don't seem to care, or don't understand how to get broader adoption. For some reason AMD's management is dead set on targeting only the high end part of the market. Like, for example, look at this blog post. Which model they're testing? DeepSeek R1, the 671B behemoth that no normal person can run. Or look at any of their tutorials/docs and s…

> For some reason AMD's management is dead set on targeting only the high end part of the market.

Because those people are dropping $100 billion on GPU clusters and individuals are not

Re: Aiter: AI Tensor Engine for ROCm

#63

Post by [deleted] »

barrenko wrote: post #6
Tpt wrote: post #4

If I understand correctly, this library provides some Torch kernels customized for AMD hardware. Why haven't they just upstreamed them to PyTorch for better adoption? Also, they seem to demo usage with Torch default eager execution mode and not Torch JIT/TorchScript. Is this library compatible with TorchScript?

I really do not understand why can't they just work with existing OSS developers pulling their hair out trying to make AMD devices work and instead do it this way. It's like Mozilla with the questionable decisions.

[deleted]

Re: Aiter: AI Tensor Engine for ROCm

#64

Post by impossiblefork »

roenxi wrote: post #14
barrenko wrote: post #6

Earlier quoted context omitted.

I really do not understand why can't they just work with existing OSS developers pulling their hair out trying to make AMD devices work and instead do it this way. It's like Mozilla with the questionable decisions.

There are a lot of OSS developers, I doubt AMD has the resources to do that. And realistically they don't need to, I wandered over to watch some George Hotz videos the other day and it looked like the AMD driver situation has improved to the point where specialist AMD access isn't needed to debug any more. Which is a huge change and very exciting for me personally because it means I might be able to jump back to an A…

There's only one Pytorch though, and it's what people are using for ML nowadays.

Back in the day you had to optimize your card for Quake, do everything to make it run well. Now you have to do that for Pytorch.

Re: Aiter: AI Tensor Engine for ROCm

#65

Post by impossiblefork »

saagarjha wrote: post #62
kouteiheika wrote: post #17

Earlier quoted context omitted.

> Why haven't they just upstreamed them to PyTorch for better adoption? They don't seem to care, or don't understand how to get broader adoption. For some reason AMD's management is dead set on targeting only the high end part of the market. Like, for example, look at this blog post. Which model they're testing? DeepSeek R1, the 671B behemoth that no normal person can run. Or look at any of their tutorials/docs and s…

> For some reason AMD's management is dead set on targeting only the high end part of the market. Because those people are dropping $100 billion on GPU clusters and individuals are not

Yes, but researchers use Pytorch and those researchers end up being the end users of the GPU clusters.

NVIDIA GPUs sell so well because they work with what researchers actually use.

Re: Aiter: AI Tensor Engine for ROCm

#67

Post by fancyfredbot »

atq2119 wrote: post #44
fancyfredbot wrote: post #23

Earlier quoted context omitted.

Notice those are all (except arguably CUDA) very mainstream languages. All four of AMDs are niche. Upstreaming this into pytorch would double the number of languages used. (Although HIP is very similar to CUDA)

HIP is essentially the same as CUDA, CK is not a language but a library, and assembly is basically used in the Nvidia ecosystem as well, in the form of PTX. There is absolutely nothing out of the ordinary here. Yes, it's multiple languages, but not any more or any different than what you'd use on an Nvidia platform (except obviously for the assembly part -- AMD's ISA is different from PTX, but that's to be expected).

I agree using both a high level and a low level language is normal, and yes using libraries is fine.

It's having both Triton and HIP in the same project which I find weird. It feels very fragmented to me to use two high level languages. Maybe it makes sense given Triton is easier to use but less fully featured, but it definitely didn't strike me as normal.

I would be interested to know if NVIDIA use more than CUDA and PTX/SASS to write CUDNN and CUBLAS.

Re: Aiter: AI Tensor Engine for ROCm

#69

Post by kllrnohj »

brrrrrm wrote: post #45
bayindirh wrote: post #28

I just want to remind everyone that El Capitan, Frontier and LUMI supercomputers are powered by AMD instinct cards. El Capitan is #1 in TOP500. Frontier is #2, LUMI is #8. ROCm development is probably mainly driven by the needs of these supercompuers' users currently. So, we're seeing the tip of the iceberg. Also ROCm packages continue to land on Debian, so there's more than meets the eye. Note: Search "AMD Instinct"…

Do super computers run in fp64 mostly? At fp8 an h100 hits 2 petaflops, and with only 1000 of them you’ve got more compute power than el capitan (in raw flop count)

It hits 2 petaflops on the tensor cores at fp8. If you want GPGPU, that plummets to 134 teraflops (for fp16, though)

Re: Aiter: AI Tensor Engine for ROCm

#70

Post by saagarjha »

impossiblefork wrote: post #65
saagarjha wrote: post #62

Earlier quoted context omitted.

> For some reason AMD's management is dead set on targeting only the high end part of the market. Because those people are dropping $100 billion on GPU clusters and individuals are not

Yes, but researchers use Pytorch and those researchers end up being the end users of the GPU clusters. NVIDIA GPUs sell so well because they work with what researchers actually use.

Oh I definitely think they should upstream to PyTorch, I'm just saying doing the usual "why doesn't AMD think of the gamers^W^W^W^W^W local model users" is not going to sway their policies.
Post reply on HN