41–50 of 222 posts
It feels like a bit of history is missing... If ollama was founded 3 years before llama.cpp was released, what engine did they use then? When did they transition?
Ollama v0.0.1 "Fast inference server written in Go, powered by llama.cpp" https://github.com/ollama/ollama/tree/v0.0.1
It feels like a bit of history is missing... If ollama was founded 3 years before llama.cpp was released, what engine did they use then? When did they transition?
Earlier quoted context omitted.
> This creates a recurring pattern on r/LocalLLaMA: new model launches, people try it through Ollama, it’s broken or slow or has botched chat templates, and the model gets blamed instead of the runtime. Seems like maybe, at least some of the time, you’re being underwhelmed my ollama not the model. The better performance point alone seems worth switching away
I follow the llama.cpp runtime improvements and it’s also true for this project. They may rush a bit less but you also have to wait for a few days after a model release to get a working runtime with most features.
For most users that wanted to run LLM locally, ollama solved the UX problem. One command, and you are running the models even with the rocm drivers without knowing. If llama provides such UX, they failed terrible at communicating that. Starting with the name. Llama.cpp: that's a cpp library! Ollama is the wrapper. That's the mental model. I don't want to build my own program! I just want to have fun :-P
No mention of the fact that Ollama is about 1000x easier to use. Llama.cpp is a great project, but it's also one of the least user friendly pieces of software I've used. I don't think anyone in the project cares about normal users. I started with Ollama, and it was great. But I moved to llama.cpp to have more up-to-date fixes. I still use Ollama to pull and list my models because it's so easy. I then built my own set…
Earlier quoted context omitted.
Llama.cpp now has a gui installed by default. It previously lacked this. Times have changed.
While that might be true, for as long as its name is “.cpp”, people are going to think it’s a C++ library and avoid it.
Earlier quoted context omitted.
Llama.cpp now has a gui installed by default. It previously lacked this. Times have changed.
While that might be true, for as long as its name is “.cpp”, people are going to think it’s a C++ library and avoid it.
In fact the first line of the wikipedia article is:
> llama.cpp is an open source software library