Ollama now supports AMD graphics cards

#1

Post by tosh » Fri, Mar 15, 2024, 5:47 PM UTC

Ollama now supports AMD graphics cards
ollama.com

Re: Ollama now supports AMD graphics cards

#2

Post by sofixa » Fri, Mar 15, 2024, 6:14 PM UTC

This is great news. The more projects do this, the less of a moat CUDA is, and the less of a competitive advantage Nvidia has.

Re: Ollama now supports AMD graphics cards

#3

Post by Symmetry » Fri, Mar 15, 2024, 6:18 PM UTC

Given the price of top line NVidia cards, if they can be had at all, there's got to be a lot of effort going on behind the scenes to improve AMD support in various places.

Re: Ollama now supports AMD graphics cards

#4

Post by [deleted] » Fri, Mar 15, 2024, 6:25 PM UTC

Symmetry wrote: post #3

Given the price of top line NVidia cards, if they can be had at all, there's got to be a lot of effort going on behind the scenes to improve AMD support in various places.

[deleted]

Re: Ollama now supports AMD graphics cards

#5

Post by anonymous-panda » Fri, Mar 15, 2024, 6:30 PM UTC

sofixa wrote: post #2

This is great news. The more projects do this, the less of a moat CUDA is, and the less of a competitive advantage Nvidia has.

What does performance look like?

Re: Ollama now supports AMD graphics cards

#6

Post by reilly3000 » Fri, Mar 15, 2024, 6:30 PM UTC

I’m curious as to how they pulled this off. OpenCL isn’t that common in the wild relative to Cuda. Hopefully it can become robust and widespread soon enough. I personally succumbed to the pressure and spent a relative fortune on a 4090 but wish I had some choice in the matter.

Re: Ollama now supports AMD graphics cards

#7

Post by deadalus » Fri, Mar 15, 2024, 6:30 PM UTC

I wish AMD did well in the Stable Diffusion front because AMD is never greedy on VRAM. The 4060Ti 16GB(minimum required for Stable Diffusion in 2024) starts at $450.

AMD with ROCm is decent on Linux but pretty bad on Windows.

Re: Ollama now supports AMD graphics cards

#8

Post by sevagh » Fri, Mar 15, 2024, 6:31 PM UTC

anonymous-panda wrote: post #5

sofixa wrote: post #2

This is great news. The more projects do this, the less of a moat CUDA is, and the less of a competitive advantage Nvidia has.

What does performance look like?

Spoiler alert: not good enough to break CUDA's moat

Re: Ollama now supports AMD graphics cards

#9

Post by Apofis » Fri, Mar 15, 2024, 6:32 PM UTC

reilly3000 wrote: post #6

I’m curious as to how they pulled this off. OpenCL isn’t that common in the wild relative to Cuda. Hopefully it can become robust and widespread soon enough. I personally succumbed to the pressure and spent a relative fortune on a 4090 but wish I had some choice in the matter.

I'm surprised they didn't speak about the implementation at all. Anyone got more intel?

Re: Ollama now supports AMD graphics cards

#10

Post by simon83 » Fri, Mar 15, 2024, 6:32 PM UTC

Does anyone know how the AMD consumer GPU support on Linux has been implemented? Must use something else than ROCm I assume? Because ROCm only supports the 7900 XTX on Linux[1], while on Windows[2] support is from RX 6600 and upwards.

[1]: https://rocblas.readthedocs.io/en/rocm-6.0.0/about/compatibi... [2]: https://rocblas.readthedocs.io/en/rocm-6.0.0/about/compatibi...