Llama.cpp supports Vulkan. why doesn't Ollama?
1–10 of 235 posts
Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#2llama.cpp has supported vulkan for more than a year now. For more than 6 months now there has been an open PR to add vulkan backend support for Ollama. However, Ollama team has not even looked at it or commented on it.
Vulkan backends are existential for running LLMs on consumer hardware (iGPUs especially). It's sad to see Ollama miss this opportunity.
Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#3llama.cpp has supported vulkan for more than a year now. For more than 6 months now there has been an open PR to add vulkan backend support for Ollama. However, Ollama team has not even looked at it or commented on it. Vulkan backends are existential for running LLMs on consumer hardware (iGPUs especially). It's sad to see Ollama miss this opportunity.
Don’t be sad for commercial entity that is not a good player https://github.com/ggerganov/llama.cpp/pull/11016#issuecomme...
Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#4llama.cpp has supported vulkan for more than a year now. For more than 6 months now there has been an open PR to add vulkan backend support for Ollama. However, Ollama team has not even looked at it or commented on it. Vulkan backends are existential for running LLMs on consumer hardware (iGPUs especially). It's sad to see Ollama miss this opportunity.
Thanks, just yesterday I discovered that Ollama could not use iGPU on my AMD machine, and was going through a long issue for solutions/workarounds (https://github.com/ollama/ollama/issues/2637). Existing instructions are based on Linux, and some people found it utterly surprising that anyone wants to run LLMs on Windows (really?). While I would have no trouble installing Linux and compile from source, I wasn't ready to do that to my main, daily-use computer.
Great to see this.
PS. Have you got feedback on whether this works on Windows? If not, I can try to create a build today.
Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#5llama.cpp has supported vulkan for more than a year now. For more than 6 months now there has been an open PR to add vulkan backend support for Ollama. However, Ollama team has not even looked at it or commented on it. Vulkan backends are existential for running LLMs on consumer hardware (iGPUs especially). It's sad to see Ollama miss this opportunity.
Don’t be sad for commercial entity that is not a good player https://github.com/ggerganov/llama.cpp/pull/11016#issuecomme...
I was not aware of this context, thanks!
Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#6Is Ollama just the porcelain around llama.cpp? Or is there more to it than that?
Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#7Is Ollama just the porcelain around llama.cpp? Or is there more to it than that?
yes, it's a convenience wrapper around llama.cpp
Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#8This is going to sound like a troll, but it's an honest question: Why do people use Ollama over llama.cpp? llama.cpp has added a ton of features, is about as user-friendly as Ollama, and is higher-performance. Is there some key differentiator for Ollama that I'm missing?
Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#9This is going to sound like a troll, but it's an honest question: Why do people use Ollama over llama.cpp? llama.cpp has added a ton of features, is about as user-friendly as Ollama, and is higher-performance. Is there some key differentiator for Ollama that I'm missing?
For starters:
- It doesn't have a website
- It doesn't have a download page, you have to build it yourself
Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#10This is going to sound like a troll, but it's an honest question: Why do people use Ollama over llama.cpp? llama.cpp has added a ton of features, is about as user-friendly as Ollama, and is higher-performance. Is there some key differentiator for Ollama that I'm missing?
https://github.com/ggerganov/llama.cpp/discussions/9565
Discussions under https://news.ycombinator.com/item?id=40693391
(I recommend doing a search yourself first)
Basically, if you know how to use a computer, you can use Ollama (almost). You can't say the same thing about llama.cpp. Not everyone knows how to build from source, or even what "build" means.