ollama is certainly a "useful" tool. I've lost count how many times my non-technical boss (a professor) said "I can run this model in ollama on my notebook, so obviously it's easy to serve and scale."
The local LLM ecosystem doesn’t need Ollama
211–220 of 222 posts
Re: The local LLM ecosystem doesn’t need Ollama
#212I've started creating https://github.com/g023/localmodelrouter/ which offers Ollama like functionality but as a single .py file with minimal dependencies and more focus on letting llama.cpp handle the dirty work.
Re: The local LLM ecosystem doesn’t need Ollama
#213thanks for opening my eyes uninstalled ollama, currently trying out LM Studio will probably try the othe ones aswell. the .cpp always scared me from using llama.cpp directly.
Re: The local LLM ecosystem doesn’t need Ollama
#214Re: The local LLM ecosystem doesn’t need Ollama
#215Earlier quoted context omitted.
And why do I use ggml-org/gemma-4-E4B-it-GGUF instead of one of the 162 other models that can be found under the ggml-org namespace? And how do I even know that this is the namespace to look at? That's what I meant by model management. I'm too tired to scroll through a bazillion models that all have very cryptic names and abbreviations just to find the one that works well on my system with my software stack. I want a…
1. Go to HF 2. Choose the model they recommend 3. Run the one-liner the site gives you Bonus: faster access to latest models and better memory usage
The first model I see on the HF homepage is this one: MiniMaxAI/MiniMax-M2.7
Do you think that this 229B parameter model will work on my consumer PC?
Stop pretending like HF is in any way beginner friendly.
Re: The local LLM ecosystem doesn’t need Ollama
#216Earlier quoted context omitted.
Yet people claim it has great UI... And still you can't define that in their GUI...
Why would you want to change a server configuration inside a client GUI? The server runs separately.
See, that’s the thing. The users don’t care or even know about the whole server/client thing. They just want to change the storage folder. A good GUI lets you do such things without the whole envvar dance.
Re: The local LLM ecosystem doesn’t need Ollama
#217Earlier quoted context omitted.
Llama.cpp now has a gui installed by default. It previously lacked this. Times have changed.
While that might be true, for as long as its name is “.cpp”, people are going to think it’s a C++ library and avoid it.
Wait, it isn't? The name very strongly suggests that it is a text file containing C++ source code; is that not the case?
Re: The local LLM ecosystem doesn’t need Ollama
#218I got tired of repeating the same points and having to dig up sources every time, so here's the timeline (as I know it) in one place with sources.
> Ollama eventually added ollama run hf.co/{repo}:{quant} to pull directly from Hugging Face, which partially addresses the availability problem. uh actually, _we_ did (generates a Docker-style manifest on the fly)
Hah! I did not know that, I'll update the article!
Re: The local LLM ecosystem doesn’t need Ollama
#219No mention of the fact that Ollama is about 1000x easier to use. Llama.cpp is a great project, but it's also one of the least user friendly pieces of software I've used. I don't think anyone in the project cares about normal users. I started with Ollama, and it was great. But I moved to llama.cpp to have more up-to-date fixes. I still use Ollama to pull and list my models because it's so easy. I then built my own set…
> so llama-swap can load Just in case you haven't seen it yet, llama.cpp now has a router mode that lets you hot-swap models. I've switched over from llama-swap and have been happy with it.
I still use llama-swap as its configuration allows me to tailor llama.cpp settings per-model, set autoload timeouts, web ui log viewing, lots of great metrics, and load/unload model on a click. Llama-swap also technically lets you port-forward to some other app/service, like a remote service.
Re: The local LLM ecosystem doesn’t need Ollama
#220Earlier quoted context omitted.
Llama.cpp now has a gui installed by default. It previously lacked this. Times have changed.
I don't care about the GUI so much. Ollama lets me download, adjust and run a whole bunch of models and they are reasonably fast. Last time I compared it with Llama.cpp, finding out how to download and install models was a pain in Llama.cpp and it was also _much_ slower than Ollama.
Having picked it up recently and compared to both llama and lm studio - the models I was using ran faster, used less memory, and had a few extra confif options available that the others hadn't implemented yet but were suggested by the model authors.
It was easy to install, run, and access the gui to get going.