71–80 of 222 posts
% ramalama run qwen3.5-9b
Error: Manifest for qwen3.5-9b:latest was not found in the Ollama registryNo mention of the fact that Ollama is about 1000x easier to use. Llama.cpp is a great project, but it's also one of the least user friendly pieces of software I've used. I don't think anyone in the project cares about normal users. I started with Ollama, and it was great. But I moved to llama.cpp to have more up-to-date fixes. I still use Ollama to pull and list my models because it's so easy. I then built my own set…
Just in case you haven't seen it yet, llama.cpp now has a router mode that lets you hot-swap models. I've switched over from llama-swap and have been happy with it.
Hmm.. pacman -Ss ollama | wc -l 16 pacman -Ss llama.cpp | wc -l 0 pacman -Ss lmstudio | wc -l 0 Maybe some day.
There are packages for Vulkan, ROCm and CUDA. They all work.
I prefer Ollama over the suggested alternatives. I will switch once we have good user experience on simple features. A new model is released on HF or the Ollama registry? One `ollama pull` and it's available. It's underwhelming? `ollama rm`.
`rm [FILE_NAME]`
With Ollama, the initial one-time setup is a little easier, and the CLI is useful, but is it worth dysfunctional templates, worse performance, and the other issues? Not to me.
Jinja templates are very common, and Jinja is not always losslessly convertible to the Go template syntax expected by Ollama. This means that some models simply cannot work correctly with Ollama. Sometimes the effects of this incompatibility are subtle and unpredictable.
Clients get disappointed, alternatives have better services, and more are popping out monthly. If they continue that way, nothing good will happen, unfortunately :(
For most users that wanted to run LLM locally, ollama solved the UX problem. One command, and you are running the models even with the rocm drivers without knowing. If llama provides such UX, they failed terrible at communicating that. Starting with the name. Llama.cpp: that's a cpp library! Ollama is the wrapper. That's the mental model. I don't want to build my own program! I just want to have fun :-P
Whip that llama! Oh wait, that's a different program.
I see no mention of vLLM in the article.
> Red Hat’s ramalama is worth a look too, a container-native model runner that explicitly credits its upstream dependencies front and center. Exactly what Ollama should have done from the start. % ramalama run qwen3.5-9b Error: Manifest for qwen3.5-9b:latest was not found in the Ollama registry
--
% ramalama run qwen3.5
> hi
Server or container exited. Shutting down client.
% ramalama run gemma4:e2b
> hello
Server or container exited. Shutting down client.
Not sure why VLC doesn't do that. It's a joke... but also not really? I mean VLC is "just" an interface to play videos. Videos are content files one "interact" with, mostly play/pause and few other functions like seeking. Because there are different video formats VLC relies on codecs to decode the videos, so basically delegating the "hard" part to codecs. Now... what's the difference here? A model is a codec, the int…
What problem is it that you are confused isn't solved? I think the codec analogy is neat but isn't the codec here llama.cpp, and the models are content files? Then the equivalent of VLC are things like LMStudio etc. which use llama.cpp to let you run models locally? I'd guess one reason we haven't solved the "codec" layer is that there doesn't seem to be a standard that open model trainers have converged on yet?
- vLLM https://vllm.ai/ ?
- oMLX https://github.com/jundot/omlx ?