Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Aiter: AI Tensor Engine for ROCm

rocm.blogs.amd.com

Post reply on HN
Print view View original

71–80 of 92 posts

Re: Aiter: AI Tensor Engine for ROCm

#71

Post by roenxi »

impossiblefork wrote: post #64
roenxi wrote: post #14

Earlier quoted context omitted.

There are a lot of OSS developers, I doubt AMD has the resources to do that. And realistically they don't need to, I wandered over to watch some George Hotz videos the other day and it looked like the AMD driver situation has improved to the point where specialist AMD access isn't needed to debug any more. Which is a huge change and very exciting for me personally because it means I might be able to jump back to an A…

There's only one Pytorch though, and it's what people are using for ML nowadays. Back in the day you had to optimize your card for Quake, do everything to make it run well. Now you have to do that for Pytorch.

> Back in the day you had to optimize your card for Quake...

That is exactly the attitude that got AMD out in the cold away from the AI revolution; they learned a lot of stupid lessons about optimising to specific games and present-day use cases instead of trying to implement general capabilities to a higher standard like Nvidia did in CUDA. They ended up a decade away from a multi-trillion dollar market

PyTorch might be special. I wouldn't be at all surprised if AMD does have a dedicated engineer working on PyTorch. But their problem to date hasn't been that their engagement with PyTorch, but rather that literally nobody could make PyTorch work on AMD cards which had buggy and terrible support for GPGPU work. If they fixed that some random might do the work without their involvement because a lot of people want to see that happen.

Re: Aiter: AI Tensor Engine for ROCm

#72

Post by Sizzle3910 »

slavik81 wrote: post #47
bayindirh wrote: post #28

I just want to remind everyone that El Capitan, Frontier and LUMI supercomputers are powered by AMD instinct cards. El Capitan is #1 in TOP500. Frontier is #2, LUMI is #8. ROCm development is probably mainly driven by the needs of these supercompuers' users currently. So, we're seeing the tip of the iceberg. Also ROCm packages continue to land on Debian, so there's more than meets the eye. Note: Search "AMD Instinct"…

> ROCm packages continue to land on Debian, so there's more than meets the eye I've been volunteering with Debian to help package ROCm for four years now, but today it officially became my full-time job. AMA.

Congrats on the job! It's exciting to see developments in CUDA competitors.

One of the issues I've had with ROCm is not so great support for commercial GPUs. This is specifically with RX 7XXX series. Do you think there is any chance it will improve in future?

Re: Aiter: AI Tensor Engine for ROCm

#73

Post by roenxi »

saagarjha wrote: post #61
bayindirh wrote: post #28

I just want to remind everyone that El Capitan, Frontier and LUMI supercomputers are powered by AMD instinct cards. El Capitan is #1 in TOP500. Frontier is #2, LUMI is #8. ROCm development is probably mainly driven by the needs of these supercompuers' users currently. So, we're seeing the tip of the iceberg. Also ROCm packages continue to land on Debian, so there's more than meets the eye. Note: Search "AMD Instinct"…

> ROCm development is probably mainly driven by the needs of these supercompuers' users currently. Seems like a problem since AMD wants to go after AI capex?

The AI capex is being invested into things that are, effectively, supercomputers.

Re: Aiter: AI Tensor Engine for ROCm

#74

Post by wkat4242 »

bayindirh wrote: post #28

I just want to remind everyone that El Capitan, Frontier and LUMI supercomputers are powered by AMD instinct cards. El Capitan is #1 in TOP500. Frontier is #2, LUMI is #8. ROCm development is probably mainly driven by the needs of these supercompuers' users currently. So, we're seeing the tip of the iceberg. Also ROCm packages continue to land on Debian, so there's more than meets the eye. Note: Search "AMD Instinct"…

They support their workstation cards pretty poorly though. I have a Radeon VII Pro and it's already deprecated in ROCm, it's not even 3 years old. They can really learn a lesson from Nvidia that supports old cards going back far and supports every card, not just a few hand-picked business models.

Re: Aiter: AI Tensor Engine for ROCm

#75

Post by cma »

saagarjha wrote: post #60
Athas wrote: post #58

Earlier quoted context omitted.

Who do you work for? And is packaging ROCm for Debian really a full-time job, or is it just a part of your job? As messy as ROCm's packaging is, I can't imagine spending all day every day trying to fix it.

> I work for AMD

from his profile, if anyone is looking for where that came from

Re: Aiter: AI Tensor Engine for ROCm

#76

Post by wewewedxfgdf »

slavik81 wrote: post #47
bayindirh wrote: post #28

I just want to remind everyone that El Capitan, Frontier and LUMI supercomputers are powered by AMD instinct cards. El Capitan is #1 in TOP500. Frontier is #2, LUMI is #8. ROCm development is probably mainly driven by the needs of these supercompuers' users currently. So, we're seeing the tip of the iceberg. Also ROCm packages continue to land on Debian, so there's more than meets the eye. Note: Search "AMD Instinct"…

> ROCm packages continue to land on Debian, so there's more than meets the eye I've been volunteering with Debian to help package ROCm for four years now, but today it officially became my full-time job. AMA.

Could you please tell AMD that it is a major competitive advantage for Nvidia that they keep doing driver updates for cards for many many years after they were released and even very old cards still get current drivers.

AMD just drops your card within a few years it seems like and drops your card from the current releases. Makes me favor Nvidia.

Re: Aiter: AI Tensor Engine for ROCm

#77

Post by saagarjha »

roenxi wrote: post #73
saagarjha wrote: post #61

Earlier quoted context omitted.

> ROCm development is probably mainly driven by the needs of these supercompuers' users currently. Seems like a problem since AMD wants to go after AI capex?

The AI capex is being invested into things that are, effectively, supercomputers.

Supercomputers have very different needs. They want 64-bit floating point which nobody has been focusing on for a while

Re: Aiter: AI Tensor Engine for ROCm

#78

Post by arcanus »

saagarjha wrote: post #77
roenxi wrote: post #73

Earlier quoted context omitted.

The AI capex is being invested into things that are, effectively, supercomputers.

Supercomputers have very different needs. They want 64-bit floating point which nobody has been focusing on for a while

While FP64 is indeed important for supercomputers, the largest supercomputers have a great deal in common with AI infrastructure.

For example, high bandwidth, low latency interconnects, supporting GPU direct network messaging and IO, are important.

High memory bandwidth is also quite important.

Debugging and performance profiling at scale also commonly uses similar tools.

Re: Aiter: AI Tensor Engine for ROCm

#79

Post by slavik81 »

wewewedxfgdf wrote: post #76
slavik81 wrote: post #47

Earlier quoted context omitted.

> ROCm packages continue to land on Debian, so there's more than meets the eye I've been volunteering with Debian to help package ROCm for four years now, but today it officially became my full-time job. AMA.

Could you please tell AMD that it is a major competitive advantage for Nvidia that they keep doing driver updates for cards for many many years after they were released and even very old cards still get current drivers. AMD just drops your card within a few years it seems like and drops your card from the current releases. Makes me favor Nvidia.

The only driver I'm aware of is the AMDGPU driver in the Linux kernel. It is updated with every release of Linux and is used for all modern AMD GPUs. I find that the drivers generally work well. My complaints are more about the user space libraries.

The good news is that I have at least one AMD GPU of each architecture from Vega to RDNA 3 / CDNA 2 on the Debian ROCm CI. Debian Trixie has packages built and tested for every modern discrete AMD GPU from Vega to RDNA 3 / CDNA 2. (I'd have liked to include RDNA 4 / CDNA 3, but the effort was quite resource constrained and the packages are a bit old. I'm hoping to improve upon that going forward, but Trixie is already in feature freeze so it will have to wait for the next release.)

I personally own much of the equipment for the Debian ROCm CI and I can promise I will continue testing new releases on old hardware for a very long time.

Re: Aiter: AI Tensor Engine for ROCm

#80

Post by slavik81 »

Athas wrote: post #58
slavik81 wrote: post #47

Earlier quoted context omitted.

> ROCm packages continue to land on Debian, so there's more than meets the eye I've been volunteering with Debian to help package ROCm for four years now, but today it officially became my full-time job. AMA.

Who do you work for? And is packaging ROCm for Debian really a full-time job, or is it just a part of your job? As messy as ROCm's packaging is, I can't imagine spending all day every day trying to fix it.

I work for AMD. To be clear, my new job is about integrating ROCm into the distribution not just about shipping ROCm packages that can run on Debian.

I'll be doing things like creating new packages in main, helping to get support for the HIP language embedded into existing dpkg tooling, helping to get GPU architecture awareness integrated into the Debian CI infrastructure, helping to enable ROCm support in other libraries and applications packaged for Debian, and ensuring that everything in Debian is successfully imported into the Ubuntu universe repositories.

Integrating HIP support into Debian so that it feels as natural as C or C++ and 'just works' across dozens of GPUs is a job for more than one person. That is why I'm glad there have been so many volunteers in the community stepping forward to help with various pieces.

Post reply on HN