Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Aiter: AI Tensor Engine for ROCm

rocm.blogs.amd.com

Post reply on HN
Print view View original

81–90 of 92 posts

Re: Aiter: AI Tensor Engine for ROCm

#81

Post by slavik81 »

Sizzle3910 wrote: post #72
slavik81 wrote: post #47

Earlier quoted context omitted.

> ROCm packages continue to land on Debian, so there's more than meets the eye I've been volunteering with Debian to help package ROCm for four years now, but today it officially became my full-time job. AMA.

Congrats on the job! It's exciting to see developments in CUDA competitors. One of the issues I've had with ROCm is not so great support for commercial GPUs. This is specifically with RX 7XXX series. Do you think there is any chance it will improve in future?

I'm not sure. What were your problems with the RX 7XXX series?

Re: Aiter: AI Tensor Engine for ROCm

#82

Post by bayindirh »

wewewedxfgdf wrote: post #76
slavik81 wrote: post #47

Earlier quoted context omitted.

> ROCm packages continue to land on Debian, so there's more than meets the eye I've been volunteering with Debian to help package ROCm for four years now, but today it officially became my full-time job. AMA.

Could you please tell AMD that it is a major competitive advantage for Nvidia that they keep doing driver updates for cards for many many years after they were released and even very old cards still get current drivers. AMD just drops your card within a few years it seems like and drops your card from the current releases. Makes me favor Nvidia.

The machine I'm writing this comment is running with a Radeon RX550, with the open source AMDGPU drivers coming with the mainline kernel.

OS is Debian Trixie (Testing). No secret sauce. Install & go. Everything is working perfectly.

Re: Aiter: AI Tensor Engine for ROCm

#83

Post by impossiblefork »

roenxi wrote: post #71
impossiblefork wrote: post #64

Earlier quoted context omitted.

There's only one Pytorch though, and it's what people are using for ML nowadays. Back in the day you had to optimize your card for Quake, do everything to make it run well. Now you have to do that for Pytorch.

> Back in the day you had to optimize your card for Quake... That is exactly the attitude that got AMD out in the cold away from the AI revolution; they learned a lot of stupid lessons about optimising to specific games and present-day use cases instead of trying to implement general capabilities to a higher standard like Nvidia did in CUDA. They ended up a decade away from a multi-trillion dollar market PyTorch migh…

Now that the required task is known though, it doesn't really matter. If AMD understand that, they should have no problem putting engineers on making Pytorch work well.

Considering its importance, it shouldn't be one engineer. It should be 50+.

Re: Aiter: AI Tensor Engine for ROCm

#84

Post by bayindirh »

saagarjha wrote: post #77
roenxi wrote: post #73

Earlier quoted context omitted.

The AI capex is being invested into things that are, effectively, supercomputers.

Supercomputers have very different needs. They want 64-bit floating point which nobody has been focusing on for a while

No they do not, because supercomputers have different partitions to cater different needs. For example, a supercomputer's half the nodes might lack a GPU to cater for the users which really need FP64 on CPU, and the other half will have GPUs for users which needs them. They will be served from different queues, so their jobs do not block each other.

OTOH, if you don't think nobody is focusing on FP64, look YoY performance gains on both CPUs and FPUs for high precision floating point performance. You'll be surprised.

Re: Aiter: AI Tensor Engine for ROCm

#85

Post by dagw »

bayindirh wrote: post #82
wewewedxfgdf wrote: post #76

Earlier quoted context omitted.

Could you please tell AMD that it is a major competitive advantage for Nvidia that they keep doing driver updates for cards for many many years after they were released and even very old cards still get current drivers. AMD just drops your card within a few years it seems like and drops your card from the current releases. Makes me favor Nvidia.

The machine I'm writing this comment is running with a Radeon RX550, with the open source AMDGPU drivers coming with the mainline kernel. OS is Debian Trixie (Testing). No secret sauce. Install & go. Everything is working perfectly.

It's not just about drivers in isolation, but what features those drivers and cards support. Support for older APIs for doing compute on AMD cards get dropped in newer drivers and newer APIs aren't supported on older cards. With Nvidia CUDA has been supported continuously for probably 15 years now, while in the AMD world you've been expected to throw out all your old code and port it to a new API every 3 years.

Re: Aiter: AI Tensor Engine for ROCm

#86

Post by manjunaths »

mjburgess wrote: post #37

Any one try any of this on a few 7900xtx (or familiarity with this hardware and platform)? I've just purchased 6 for some small-scale experimentation. I'm thinking the next machine I'll use AMD Radeon PRO W7900 (to get 128 GB VRAM / machine).

I have a 7900 GRE, which is the same except less memory. I run Gemma 3, LLama 3.1, the QwQ models and the DeepSeek distilled models using llama.cpp. They run fine, I especially like the new Gemma3-27b-Q6 (20 GB model), I get 2 tok/s on it.

I have also run Hunyuan3d-2 and generated 3d models. You would've to separate out the model generation and texture generation phase, but it works.

I run ComfyUI and bootleg gguf models. This is all on windows. Now even WSL2 works, so I am using Ubuntu-24.04 on Windows 11 to run Hunyuan3D-2.

For LLMs, llama.cpp native binaries are available. Everything just works out of the box.

Re: Aiter: AI Tensor Engine for ROCm

#87

Post by slavik81 »

slavik81 wrote: post #79
wewewedxfgdf wrote: post #76

Earlier quoted context omitted.

Could you please tell AMD that it is a major competitive advantage for Nvidia that they keep doing driver updates for cards for many many years after they were released and even very old cards still get current drivers. AMD just drops your card within a few years it seems like and drops your card from the current releases. Makes me favor Nvidia.

The only driver I'm aware of is the AMDGPU driver in the Linux kernel. It is updated with every release of Linux and is used for all modern AMD GPUs. I find that the drivers generally work well. My complaints are more about the user space libraries. The good news is that I have at least one AMD GPU of each architecture from Vega to RDNA 3 / CDNA 2 on the Debian ROCm CI. Debian Trixie has packages built and tested for…

This is the hardware we test on: https://salsa.debian.org/rocm-team/community/team-project/-/...

Re: Aiter: AI Tensor Engine for ROCm

#88

Post by atq2119 »

fancyfredbot wrote: post #67
atq2119 wrote: post #44

Earlier quoted context omitted.

HIP is essentially the same as CUDA, CK is not a language but a library, and assembly is basically used in the Nvidia ecosystem as well, in the form of PTX. There is absolutely nothing out of the ordinary here. Yes, it's multiple languages, but not any more or any different than what you'd use on an Nvidia platform (except obviously for the assembly part -- AMD's ISA is different from PTX, but that's to be expected).

I agree using both a high level and a low level language is normal, and yes using libraries is fine. It's having both Triton and HIP in the same project which I find weird. It feels very fragmented to me to use two high level languages. Maybe it makes sense given Triton is easier to use but less fully featured, but it definitely didn't strike me as normal. I would be interested to know if NVIDIA use more than CUDA an…

I would argue that Triton is in fact higher-level than HIP. Plus, it is more specialised for specific use cases.

Re: Aiter: AI Tensor Engine for ROCm

#89

Post by clayhacks »

slavik81 wrote: post #81
Sizzle3910 wrote: post #72

Earlier quoted context omitted.

Congrats on the job! It's exciting to see developments in CUDA competitors. One of the issues I've had with ROCm is not so great support for commercial GPUs. This is specifically with RX 7XXX series. Do you think there is any chance it will improve in future?

I'm not sure. What were your problems with the RX 7XXX series?

Not the GP, but I have a RX 7700S running Ubuntu and I cannot for the life of me get ROCm to play nice with my GPU. I tried all sorts of env vars but I keep getting seg faults when I try to run PyTorch. Or it just ends up running on my CPU

Re: Aiter: AI Tensor Engine for ROCm

#90

Post by slavik81 »

clayhacks wrote: post #89
slavik81 wrote: post #81

Earlier quoted context omitted.

I'm not sure. What were your problems with the RX 7XXX series?

Not the GP, but I have a RX 7700S running Ubuntu and I cannot for the life of me get ROCm to play nice with my GPU. I tried all sorts of env vars but I keep getting seg faults when I try to run PyTorch. Or it just ends up running on my CPU

The RX 7700S is gfx1102. Please see my reply in the thread on the RX 7600, as it is applicable to you too. https://news.ycombinator.com/item?id=43465281
Post reply on HN