Ollama needs competition. I’m not sure what drives the people that maintain it but some of their actions imply that there are ulterior motives at play that do not have the benefit of their users in mind. However such projects require a lot of time and effort and it’s not clear if this project can be forked and kept alive.
Llama.cpp supports Vulkan. why doesn't Ollama?
41–50 of 235 posts
Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#42Ollama is sketchy enough that I run it in a VM. Which is odd because it would probably take less effort to just run Llama.cpp directly, but VMs are pretty easy so just went that route. When I see people bring up the sketchiness most of the time the creator responds with the equivalent of shrugs , which imo increases the sketchiness.
ollama advertising llama.cpp features as their own is very dishonest in my opinion.
Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#43Earlier quoted context omitted.
For starters: - It doesn't have a website - It doesn't have a download page, you have to build it yourself
> - It doesn't have a download page, you have to build it yourself I'd wager that anyone capable enough to run a command line tool like Ollama should also be able to download prebuilt binaries from the llama.cpp releases page[1]. Also, prebuilt binaries are available on things like homebrew[2]. [1]: https://github.com/ggerganov/llama.cpp/releases [2]: https://formulae.brew.sh/formula/llama.cpp
Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#44Can someone explain what the point of ollama is? Every time I look at it, it seems like it's a worse llama.cpp that removes options to make things "easier".
Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#45Ollama tries to appeal to a lowest common denominator user base, who does not want to worry about stuff like configuration and quants, or which binary to download. I think they want their project to be smart enough to just 'figure out what to do' on behalf of the user. That appeals to a lot of people, but I think them stuffing all backends into one binary and auto-detecting at runtime which to use and is actually a s…
This PR is #1 on their repo based on multiple metrics (comments, iterations, what have you)
Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#46Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#47I don't know why one would use Ollama instead of llama.cpp. llama.cpp is so easy to use and the maintainer is pretty famous and active in the community.
Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#48Earlier quoted context omitted.
Don’t be sad for commercial entity that is not a good player https://github.com/ggerganov/llama.cpp/pull/11016#issuecomme...
This is great, I did not know about RamaLama and I'll be using and recommending that in future and if I see people using Ollama in instructions I'll recommend they move to RamaLama in the future. Cheers.
As usual, the real work seems to be appropriated by people who do the last little bit — put an acceptable user experience and some polish on it — and they take all the money and credit.
It’s shitty but it also happens because the vast majority of devs, especially in the FOSS world, do not understand or appreciate user experience. It is bar none the most important thing in the success of most things in computing.
My rule is: every step a user has to do to install or set up something halves adoption. So if 100 people enter and there are two steps, 25 complete the process.
For a long time Apple was the most valuable corporation on Earth on the basis of user experience alone. Apple doesn’t invent much. They polish it, and that’s where like 99% of the value is as far as the market is concerned.
The reason is that computers are very confusing and hard to use. Computer people, which most of us are, don’t see that because it’s second nature to us. But even for computer people you get to the point where you’re busy and don’t have time to nerd out on every single thing you use, so it even matters to computer people in the end.
Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#49Ollama is sketchy enough that I run it in a VM. Which is odd because it would probably take less effort to just run Llama.cpp directly, but VMs are pretty easy so just went that route. When I see people bring up the sketchiness most of the time the creator responds with the equivalent of shrugs , which imo increases the sketchiness.
Don’t you need at least 2 GPUs in that case and put kernel level passthrough?
Re: Llama.cpp supports Vulkan. why doesn't Ollama?
#50Ollama is sketchy enough that I run it in a VM. Which is odd because it would probably take less effort to just run Llama.cpp directly, but VMs are pretty easy so just went that route. When I see people bring up the sketchiness most of the time the creator responds with the equivalent of shrugs , which imo increases the sketchiness.
It's fully open source. I mean yes it uses llama.cpp without giving it credit. But why run it in a VM?