Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The local LLM ecosystem doesn’t need Ollama

sleepingrobots.com

Post reply on HN
Print view View original

171–180 of 222 posts

Re: The local LLM ecosystem doesn’t need Ollama

#171

Post by JKCalhoun »

cientifico wrote: post #25

For most users that wanted to run LLM locally, ollama solved the UX problem. One command, and you are running the models even with the rocm drivers without knowing. If llama provides such UX, they failed terrible at communicating that. Starting with the name. Llama.cpp: that's a cpp library! Ollama is the wrapper. That's the mental model. I don't want to build my own program! I just want to have fun :-P

"LM Studio… Jan… Msty… koboldcpp…"

Plenty of alternatives listed. Can anyone with experience suggest the likely successor to Ollama? I have a Mac Mini but don't mind a C/L tool.

I think, as was pointed out, Ollama won because of how easy it is to set up, pull down new models. I would expect similar for a replacement.

Re: The local LLM ecosystem doesn’t need Ollama

#173

Post by justinclift »

adrian_b wrote: post #169
AndroTux wrote: post #28

Earlier quoted context omitted.

Exactly. The blog post states that the alternatives listed are similarly intuitive. They are not. If you just need a chat app, then sure, there’s plenty of options. But if you want an OpenAI compatible API with model management, accessibility breaks down fast. I’m open to suggestions, but the alternatives outlined in the blog post ain’t it.

What you say was true in the past. As other posters report, now llama-server implements an OpenAI compatible API and you can also connect to it with any Web browser. I have not tried yet the OpenAI API, but it should have eliminated the last Ollama advantage. I do not believe that the Ollama "curated" models are significantly easier to use for a newbie than downloading the models directly from Huggingface. On Hugging…

> On Huggingface you have much more details about models...

For a (brand new!) newbie, it's very, very likely to be information overload.

They're still at the start of their journey, so simple tends to be better for 90% of users. ;)

Re: The local LLM ecosystem doesn’t need Ollama

#175

Post by kashyapc »

0xbadcafebee wrote: post #13

No mention of the fact that Ollama is about 1000x easier to use. Llama.cpp is a great project, but it's also one of the least user friendly pieces of software I've used. I don't think anyone in the project cares about normal users. I started with Ollama, and it was great. But I moved to llama.cpp to have more up-to-date fixes. I still use Ollama to pull and list my models because it's so easy. I then built my own set…

> No mention of the fact that Ollama is about 1000x easier to use.

The point of the article is not to expound on how user-friendly "Ollama" is. It's about exposing the deception and shameful moral low ground they took.

Re: The local LLM ecosystem doesn’t need Ollama

#176

Post by kashyapc »

Zetaphor wrote: post #2

I got tired of repeating the same points and having to dig up sources every time, so here's the timeline (as I know it) in one place with sources.

Thank you; it's an educating read for me, as someone who doesn't dwell in this space, but cares about FOSS in its true spirit.

Re: The local LLM ecosystem doesn’t need Ollama

#177

Post by erusev »

This is partly why we're building LlamaBarn. It's a lightweight macOS menu bar app that runs llama-server under the hood, with models stored as standard GGUFs in your Hugging Face cache — the same location llama-server uses by default. No separate model store, no lock-in.

https://github.com/ggml-org/LlamaBarn

Re: The local LLM ecosystem doesn’t need Ollama

#178

Post by AndroTux »

Philip-J-Fry wrote: post #144
AndroTux wrote: post #28

Earlier quoted context omitted.

Exactly. The blog post states that the alternatives listed are similarly intuitive. They are not. If you just need a chat app, then sure, there’s plenty of options. But if you want an OpenAI compatible API with model management, accessibility breaks down fast. I’m open to suggestions, but the alternatives outlined in the blog post ain’t it.

What do you mean? LMStudio is listed as an alternative. It offers a chat UI, a model server supporting OpenAI, Anthropic and LMStudio API interfaces. It supports loading the models on demand or picking what models you want loaded. And you can tweak every parameter. And it uses llama.cpp which is the whole point of the blog post.

Thanks for pointing that out. From the description in the blog post it sounded like it was GUI only without an API, and I didn't bother looking into it because of that. But it look pretty nice, so I'll give it a try.

Re: The local LLM ecosystem doesn’t need Ollama

#179

Post by AndroTux »

homarp wrote: post #65
AndroTux wrote: post #28

Earlier quoted context omitted.

Exactly. The blog post states that the alternatives listed are similarly intuitive. They are not. If you just need a chat app, then sure, there’s plenty of options. But if you want an OpenAI compatible API with model management, accessibility breaks down fast. I’m open to suggestions, but the alternatives outlined in the blog post ain’t it.

like someone said above: brew install llama.cpp llama-server -hf ggml-org/gemma-4-E4B-it-GGUF --port 8000 (with MCP support and web chat interface) and you have OpenAI API on the same 8000 port. ( https://github.com/ggml-org/llama.cpp/tree/master/tools/serv... lists the endpoints)

And why do I use ggml-org/gemma-4-E4B-it-GGUF instead of one of the 162 other models that can be found under the ggml-org namespace? And how do I even know that this is the namespace to look at?

That's what I meant by model management. I'm too tired to scroll through a bazillion models that all have very cryptic names and abbreviations just to find the one that works well on my system with my software stack.

I want a simple interface that a tool like me can scroll through easily, click on, and then have a model that works well enough. If I put in that much brain power to get my LLM working, I might as well do the work myself instead of using an LLM in the first place.

Re: The local LLM ecosystem doesn’t need Ollama

#180

Post by Eisenstein »

cyanydeez wrote: post #161
Eisenstein wrote: post #155

Earlier quoted context omitted.

Its not possible to run the latest model architectures without 'moving fast'. The only thing broken here is that they are trying to use an old version with a new model.

and Ollama suffered the same fate when wanting to try new models

What fate?
Post reply on HN