111–120 of 133 posts
[flagged]
> Been running local LLMs on my 7900 XTX for months and the ROCm experience has been... rough. Just out of curiosity... how so? I only ask because I've been running local models (using Ollama) on my RX 7900 XTX for the last year and a half or so and haven't had a single problem that was ROCm specific that I can think of. Actually, I've barely had any problems at all, other than the card being limited to 24GB of VRAM.…
Maybe it's a language barrier problem, but "by AMD" makes me think its a project distributed by AMD. Is that actually the case? I'm not seeing any reason to believe it is.
Earlier quoted context omitted.
> Been running local LLMs on my 7900 XTX for months and the ROCm experience has been... rough. Just out of curiosity... how so? I only ask because I've been running local models (using Ollama) on my RX 7900 XTX for the last year and a half or so and haven't had a single problem that was ROCm specific that I can think of. Actually, I've barely had any problems at all, other than the card being limited to 24GB of VRAM.…
Did you have complete hardware lockups when VRAM is exceeded? I had quite a few on my 7900XTX with llama.cpp (Arch Linux, various driver versions). Once I dial in the quant and context size that never exceed VRAM, it is stable; before that I swear a lot and keep pressing the hardware reset button.
I've had a few of those "model psychosis" incidents where the context gets so big that the model just loses all coherence and starts spewing gibberish though. Those are always fun.
Earlier quoted context omitted.
You have got to be joking. My three NVIDIA cards are more power efficient than my one AMD card, both at idle and during usage. Official ROCm is like pulling teeth with poor support for desktop cards. Debian, a volunteer led project, have better ROCm CI than AMD and support more cards. Look at any benchmarks. NV midrange cards are faster than AMD and at least a generation in front. Owning a 7900XTX is an embarrassing…
> Official ROCm is like pulling teeth with poor support for desktop cards... I agree with most of your post and fled the AMD ecosystem some time ago because of the machine learning situation, but their problem seemed to be more the firmware bugs and memory management of compute shaders than the higher level libraries. The obvious solution to this one would be not to use ROCm. ROCm has always been a bit of a train wre…
However for pp, Vulkan is still nowhere near close to ROCm. That matters for long context and/or quick response. A lot of people really care about that time-to-first-token.
Earlier quoted context omitted.
> Been running local LLMs on my 7900 XTX for months and the ROCm experience has been... rough. Just out of curiosity... how so? I only ask because I've been running local models (using Ollama) on my RX 7900 XTX for the last year and a half or so and haven't had a single problem that was ROCm specific that I can think of. Actually, I've barely had any problems at all, other than the card being limited to 24GB of VRAM.…
Did you have complete hardware lockups when VRAM is exceeded? I had quite a few on my 7900XTX with llama.cpp (Arch Linux, various driver versions). Once I dial in the quant and context size that never exceed VRAM, it is stable; before that I swear a lot and keep pressing the hardware reset button.
Earlier quoted context omitted.
I exclusively buy AMD hardware for local inference. For open drivers, power efficiency, and cost AMD beats Nvidia easily for consumers.
You have got to be joking. My three NVIDIA cards are more power efficient than my one AMD card, both at idle and during usage. Official ROCm is like pulling teeth with poor support for desktop cards. Debian, a volunteer led project, have better ROCm CI than AMD and support more cards. Look at any benchmarks. NV midrange cards are faster than AMD and at least a generation in front. Owning a 7900XTX is an embarrassing…
Earlier quoted context omitted.
As another data point. Running Qwen3.5 122B at 35t/s as a daily driver using Vulcan llama.cpp on kernel 7.0.0rc5 on a Framework Desktop board (Strix Halo 128). Also a pair of AMD AI Pro r9700 cards as my workhorses for zimageturbo, qwen tts/asr and other accessory functions and experiments. Finally have a Radeon 6900 XT running qwen3.5 32B at 60+t/s for a fast all arounder. If I buy anything nvidia it will be only fo…
How is the performance for Z-Image on the R9700s?
Earlier quoted context omitted.
As another data point. Running Qwen3.5 122B at 35t/s as a daily driver using Vulcan llama.cpp on kernel 7.0.0rc5 on a Framework Desktop board (Strix Halo 128). Also a pair of AMD AI Pro r9700 cards as my workhorses for zimageturbo, qwen tts/asr and other accessory functions and experiments. Finally have a Radeon 6900 XT running qwen3.5 32B at 60+t/s for a fast all arounder. If I buy anything nvidia it will be only fo…
Are the dedicated GPU cards on another machine or you’re using eGPU with the framework?
Earlier quoted context omitted.
I exclusively buy AMD hardware for local inference. For open drivers, power efficiency, and cost AMD beats Nvidia easily for consumers.
Any recommendations in the current market? Love how plug and play and is on Linux from the driver side of things.