161–170 of 222 posts
Earlier quoted context omitted.
And that's exactly why llama.cpp is not usable by casual users. They follow the "move fast and break things" model. With ollama, you just have to make sure you're getting/building the latest version.
Its not possible to run the latest model architectures without 'moving fast'. The only thing broken here is that they are trying to use an old version with a new model.
ollama is pretty intuitive to use still - dont see why will stop.
I guess if you're not frustrated with things like this then sure, no need to stop using it.
I got tired of repeating the same points and having to dig up sources every time, so here's the timeline (as I know it) in one place with sources.
uh actually, _we_ did (generates a Docker-style manifest on the fly)
I'm sorry, on a mac, Ollama just works. It lets me use a model and test it quickly. This is like saying stop using google drive, upload everything to s3 instead! When i'm using Ollama - I honeslty don't care about performance, I'm looking to try out a model and then if it seems good, place it onto a most dedicated stack specifically for it.
Ollama is a bit easier to use, you’re right. But the point of the article is the way they just disregarded the license of llama.cpp, moved away from open source while still claiming to be open source and pivoted to cloud offerings when the whole point was to run local models all while without contributing anything back to the big open source projects it owns its existence to. Maybe you don’t care about performance (w…
Currently i've found Ollama to have the best intuitive experience for trying new models. Once i've tried those models and decide on something to use for a project, I can deploy them, and not need to use a UI again.
I'll be trying out the other options in this thread, but my point is that ease of use is going to triumph over the other points the original post made, and some of the alternatives mentioned in the original post miss why Ollama is so popular.
Ah man the VC death trap. It's ok. I don't mean it like that but this is classic. It's unavoidable. They gotta make money. They took money, they gotta make money. It's not easy. Everyone has principles, developers more than anyone. They are developers, they are people like you and me. They didn't even start as ollama. They started as a kubernetes infra project in YC and pivoted. Listen don't be hard on these guys. It…
> This stuff is messy. Give them some credit. They give you an epic open source project. It seems to me the epic open source project was given to us by Georgi Gerganov. These people just tried to milk it for some money, and made everything a little worse in the process.
UX is where the money is, it is in the wrapper, not the core.
Unfortunately, the core is the most valuable and labor intensive part of it.
With agentic coding, the gap between solid core and shitty wrapper is going to be wider and wider.
Earlier quoted context omitted.
Having read above article, I just gave llama.cpp a shot. It is as easy as the author says now, though definitely not documented quite as well. My quickstart: brew install llama.cpp llama-server -hf ggml-org/gemma-4-E4B-it-GGUF --port 8000 Go to localhost:8000 for the Web UI. On Linux it accelerates correctly on my AMD GPU, which Ollama failed to do, though of course everyone's mileage seems to vary on this.
Was hoping it was so easy :) But I probably need to look into it some more. llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'gemma4' llama_model_load_from_file_impl: failed to load model Edit: @below, I used `nix-shell -p llama-cpp` so not brew related. Could indeed be an older version indeed! I'll check.
There are 2 main reasons. One is the tokenizer, where new tokenizer definitions may be mishandled by the older tokenizer parsers.
The second reason is that each model may implement differently the tool invocations, e.g. by using different delimiter tokens and different text layouts for describing the parameters of a tool invocation.
Therefore running the Gemma-4 models encountered various problems during the first days after their release, especially for the dense 31B model.
Solving these problems required both a new version of llama.cpp (also for other inference backends) and updates in the model chat template and tokenizer configuration files.
So anyone who wants to use Gemma-4 should update to the latest version of llama.cpp and to the latest models from Huggingface, because the latest updates have been a couple of days ago.
Earlier quoted context omitted.
It would make sense to just make the GUI a separate project, they could call it llama.gui.
it is called llama-barn https://github.com/ggml-org/LlamaBarn
On non-Apple PCs, "llama-server" is what you use, and you can connect to it either with a browser or with an application compatible with the OpenAI API.
Perhaps using "llama-server" as the name of the project would have been less confusing for newbies than "llama.cpp".
I confess that when I first heard about "llama.cpp" I also thought that it is just a library and that I have to write my own program in order to implement a complete LLM inference backend.
Earlier quoted context omitted.
Ollama is a bit easier to use, you’re right. But the point of the article is the way they just disregarded the license of llama.cpp, moved away from open source while still claiming to be open source and pivoted to cloud offerings when the whole point was to run local models all while without contributing anything back to the big open source projects it owns its existence to. Maybe you don’t care about performance (w…
I'm making more of a pragmatic point. While ethics of companies are important, i'm still using OpenAI, Anthropic, Microsoft, Apple etc, so I definitely accept a trade-off between morality and ease-of-use. Currently i've found Ollama to have the best intuitive experience for trying new models. Once i've tried those models and decide on something to use for a project, I can deploy them, and not need to use a UI again.…
No mention of the fact that Ollama is about 1000x easier to use. Llama.cpp is a great project, but it's also one of the least user friendly pieces of software I've used. I don't think anyone in the project cares about normal users. I started with Ollama, and it was great. But I moved to llama.cpp to have more up-to-date fixes. I still use Ollama to pull and list my models because it's so easy. I then built my own set…
Exactly. The blog post states that the alternatives listed are similarly intuitive. They are not. If you just need a chat app, then sure, there’s plenty of options. But if you want an OpenAI compatible API with model management, accessibility breaks down fast. I’m open to suggestions, but the alternatives outlined in the blog post ain’t it.
As other posters report, now llama-server implements an OpenAI compatible API and you can also connect to it with any Web browser.
I have not tried yet the OpenAI API, but it should have eliminated the last Ollama advantage.
I do not believe that the Ollama "curated" models are significantly easier to use for a newbie than downloading the models directly from Huggingface.
On Huggingface you have much more details about models, which can allow you to navigate through the jungle of countless model variants, to find what should be more suitable for yourself.
The fact criticized in TFA, that the Ollama "curated" list can be misleading about the characteristics of the models, is a very serious criticism from my point of view, which is enough for me to not use such "curated" models.
I am not aware of any alternative for choosing and downloading the right model for local inference that is superior to using directly the Huggingface site.
I believe that choosing a model is the most intimidating part for a newbie who wants to run inference locally.
If a good choice is made, downloading the model, installing llama.cpp and running llama-server are trivial actions, which require minimal skills.
No mention of the fact that Ollama is about 1000x easier to use. Llama.cpp is a great project, but it's also one of the least user friendly pieces of software I've used. I don't think anyone in the project cares about normal users. I started with Ollama, and it was great. But I moved to llama.cpp to have more up-to-date fixes. I still use Ollama to pull and list my models because it's so easy. I then built my own set…
Agreed ollama is a good intro but once you move beyond it starts to be a pain.