41–50 of 234 posts
Earlier quoted context omitted.
What does performance look like?
Spoiler alert: not good enough to break CUDA's moat
Still, I think AMD will catch or overtake NVidia in hardware soon, but software is a bigger problem. Hopefully the opensource strategy will pay off for them.
It's pretty funny to see this blog post, when I have been running Ollama on my AMD RX 6650 for weeks :D They have shipped ROCm containers since 0.1.27 (21 days ago). This blog post seems to be published along with the latest release, 0.1.29. I wonder what they actually changed in this release with regards to AMD support. Also: see this issue[0] that I made where I worked through running Ollama on an AMD card that the…
Earlier quoted context omitted.
Super easy. You can just head down to https://lmstudio.ai and pick up an app that lets you play around. It's not particularly advanced, but it works pretty well. It's mostly optimized for M-series silicon, but it also technically works on Windows, and isn't too difficult to trick into working on Linux either.
Also, https://jan.ai is open source and worth trying out too.
I wonder why they aren't supporting RX 6750 XT and lower yet, are there architectural differences between these and RX 6800+?
LLVM has recently introduced a unified ISA for all RDNA 2 GPUs (gfx10.3-generic), so the need for the environment variable workaround mentioned in the other comment should eventually disappear.
It's pretty funny to see this blog post, when I have been running Ollama on my AMD RX 6650 for weeks :D They have shipped ROCm containers since 0.1.27 (21 days ago). This blog post seems to be published along with the latest release, 0.1.29. I wonder what they actually changed in this release with regards to AMD support. Also: see this issue[0] that I made where I worked through running Ollama on an AMD card that the…
I mean, it was 21 days ago. What’s the difference?
I’m curious as to how they pulled this off. OpenCL isn’t that common in the wild relative to Cuda. Hopefully it can become robust and widespread soon enough. I personally succumbed to the pressure and spent a relative fortune on a 4090 but wish I had some choice in the matter.
Does anyone know how the AMD consumer GPU support on Linux has been implemented? Must use something else than ROCm I assume? Because ROCm only supports the 7900 XTX on Linux[1], while on Windows[2] support is from RX 6600 and upwards. [1]: https://rocblas.readthedocs.io/en/rocm-6.0.0/about/compatibi... [2]: https://rocblas.readthedocs.io/en/rocm-6.0.0/about/compatibi...
I wouldn't read too much into support. It's more in terms of business/warranty/promises than what can actually do things I've had a 6900XT since launch and this is the first I'm hearing "unsupported" , having played with ROCM plenty over the years with Fedora Linux. I think, at most, it's taken a couple key environment variables
I'm not sure what is worse from AMD - the ML software support they provide for their cards, or the utterly crap documentation.
How about one page documenting AMD's software stack compared to NVIDIA, one page documenting what ML frameworks support AMD cards, and another documenting "compute capability" type numbers to define the capabilities of different cards.
I wish AMD did well in the Stable Diffusion front because AMD is never greedy on VRAM. The 4060Ti 16GB(minimum required for Stable Diffusion in 2024) starts at $450. AMD with ROCm is decent on Linux but pretty bad on Windows.
They bump up VRAM because they can't compete on raw compute.
Many thanks to ollama project and llama.cpp!