i had no idea about all this. especially the performance and bugs. thanks for informing me!
The local LLM ecosystem doesn’t need Ollama
11–20 of 222 posts
Re: The local LLM ecosystem doesn’t need Ollama
#12I prefer Ollama over the suggested alternatives.
I will switch once we have good user experience on simple features.
A new model is released on HF or the Ollama registry? One `ollama pull` and it's available. It's underwhelming? `ollama rm`.
Re: The local LLM ecosystem doesn’t need Ollama
#13No mention of the fact that Ollama is about 1000x easier to use. Llama.cpp is a great project, but it's also one of the least user friendly pieces of software I've used. I don't think anyone in the project cares about normal users.
I started with Ollama, and it was great. But I moved to llama.cpp to have more up-to-date fixes. I still use Ollama to pull and list my models because it's so easy. I then built my own set of scripts to populate a separate cache directory of hardlinks so llama-swap can load the gguf's into llama.cpp.
Re: The local LLM ecosystem doesn’t need Ollama
#14I think the biggest advantage for me with ollama is the ability to "hotswap" models with different utility instead of restarting the server with different models combined with the simple "ollama pull model". In other words, it has been quite convenient. Due to this post I had to search a bit and it seems that llama.cpp recently got router support[1], so I need to have a look at this. My main use for this is a discord…
> the ability to "hotswap" models with different utility instead of restarting the server
The article mentions llama-swap does this
Re: The local LLM ecosystem doesn’t need Ollama
#15I prefer Ollama over the suggested alternatives. I will switch once we have good user experience on simple features. A new model is released on HF or the Ollama registry? One `ollama pull` and it's available. It's underwhelming? `ollama rm`.
> This creates a recurring pattern on r/LocalLLaMA: new model launches, people try it through Ollama, it’s broken or slow or has botched chat templates, and the model gets blamed instead of the runtime.
Seems like maybe, at least some of the time, you’re being underwhelmed my ollama not the model.
The better performance point alone seems worth switching away
Re: The local LLM ecosystem doesn’t need Ollama
#16It feels like a bit of history is missing... If ollama was founded 3 years before llama.cpp was released, what engine did they use then? When did they transition?
Re: The local LLM ecosystem doesn’t need Ollama
#17The performance issues are crazy. Thanks for sharing this
Re: The local LLM ecosystem doesn’t need Ollama
#18i use goose by block
seems pretty unrelated to the post?
also you might be the only person in the wild I've seen admit to this
Re: The local LLM ecosystem doesn’t need Ollama
#19I got tired of repeating the same points and having to dig up sources every time, so here's the timeline (as I know it) in one place with sources.
Thanks for writing this, I hope people here will actually read this and not assume this is some unfounded hit piece. I was involved a little bit in llama.cpp and knew most of what you wrote and it’s just disgusting how ollama founders behaved!
For people looking for alternatives, I would also recommend llama-file, it’s a one file executable for any OS that includes your chosen model: https://github.com/mozilla-ai/llamafile?tab=readme-ov-file
It’s truly open source, backed by Mozilla, openly uses llama.cpp and was created by wizard Justine Tunney of CosmopolitanC fame.
Re: The local LLM ecosystem doesn’t need Ollama
#20I noticed the performance issues too. I started using Jan recently and tried running the same model via llama.cpp vs local ollama, and the llama.cpp one was noticeably faster.