Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The local LLM ecosystem doesn’t need Ollama

sleepingrobots.com

Post reply on HN
Print view View original

141–150 of 222 posts

Re: The local LLM ecosystem doesn’t need Ollama

#141

Post by damnitbuilds »

yuppiepuppie wrote: post #92
damnitbuilds wrote: post #86

Earlier quoted context omitted.

The author points out that the Ollama people are evil. So it is more like saying "Stop using SCO Unix, use Linux instead".

Where do they use the term "evil"?

They might not use the word, but the behavior they describe is evil:

" This isn’t a matter of open-source etiquette, the MIT license has exactly one major requirement: include the copyright notice. Ollama didn’t.

The community noticed. GitHub issue #3185 was opened in early 2024 requesting license compliance. It went over 400 days without a response from maintainers. When issue #3697 was opened in April 2024 specifically requesting llama.cpp acknowledgment, community PR #3700 followed within hours. Ollama’s co-founder Michael Chiang eventually added a single line to the bottom of the README: “llama.cpp project founded by Georgi Gerganov.” "

Re: The local LLM ecosystem doesn’t need Ollama

#142

Post by omgitspavel »

cientifico wrote: post #25

For most users that wanted to run LLM locally, ollama solved the UX problem. One command, and you are running the models even with the rocm drivers without knowing. If llama provides such UX, they failed terrible at communicating that. Starting with the name. Llama.cpp: that's a cpp library! Ollama is the wrapper. That's the mental model. I don't want to build my own program! I just want to have fun :-P

agree. We can easily compare it with docker. Of course people can use runc directly, but most people select not to and use `docker run` instead.

And you can blame docker in a similar manner. LXC existed for at least 5 years before docker. But docker was just much more convenient to use for an average user.

UX is a huge factor for adoption of technology. If a project fails at creating the right interface, there is nothing wrong with creating a wrapper.

Re: The local LLM ecosystem doesn’t need Ollama

#143

Post by zozbot234 »

ekianjo wrote: post #135
zozbot234 wrote: post #60

Earlier quoted context omitted.

> Do they still not let you change the default model folder? Actually they do. It's environment variable OLLAMA_MODELS in the server configuration file.

Yet people claim it has great UI... And still you can't define that in their GUI...

Why would you want to change a server configuration inside a client GUI? The server runs separately.

Re: The local LLM ecosystem doesn’t need Ollama

#144

Post by Philip-J-Fry »

AndroTux wrote: post #28
0xbadcafebee wrote: post #13

No mention of the fact that Ollama is about 1000x easier to use. Llama.cpp is a great project, but it's also one of the least user friendly pieces of software I've used. I don't think anyone in the project cares about normal users. I started with Ollama, and it was great. But I moved to llama.cpp to have more up-to-date fixes. I still use Ollama to pull and list my models because it's so easy. I then built my own set…

Exactly. The blog post states that the alternatives listed are similarly intuitive. They are not. If you just need a chat app, then sure, there’s plenty of options. But if you want an OpenAI compatible API with model management, accessibility breaks down fast. I’m open to suggestions, but the alternatives outlined in the blog post ain’t it.

What do you mean?

LMStudio is listed as an alternative. It offers a chat UI, a model server supporting OpenAI, Anthropic and LMStudio API interfaces. It supports loading the models on demand or picking what models you want loaded. And you can tweak every parameter.

And it uses llama.cpp which is the whole point of the blog post.

Re: The local LLM ecosystem doesn’t need Ollama

#145

Post by teekert »

homarp wrote: post #124
teekert wrote: post #114

Earlier quoted context omitted.

Ah it was indeed just that! I'm now on: $ llama --version version: 8770 (82764d8) built with GNU 15.2.0 for Linux x86_64 (From Nix unstable) And this works as advertised, nice chat interface, but no openai API I guess, so no opencode...

check on same port, there is an OpenAI API https://github.com/ggml-org/llama.cpp/tree/master/tools/serv...

Good stuff, thanx!

Re: The local LLM ecosystem doesn’t need Ollama

#146

Post by WobblyDev »

maybewhenthesun wrote: post #128
u1hcw9nx wrote: post #118

Two Views of MIT-Style Licenses: 1. MIT-style licenses are "do what you want" as long as you provide a single line of attribution. Including building big closed source business around it. 2. MIT-style licenses are "do what you want" under the law, but they carry moral, GPL-like obligations to think about the "community." To my knowledge Georgi Gerganov, the creator of llama.cpp, has only complained about attribution…

The second interpretation is nonsense of course. If you want GPL-like obligations, use the GPL. A license is what it says in the license, nothing extra. It's a legal document not a moral guideline. I do think it's a very good idea to always use the GPL (even though commercially minded types always get their panties in a bunch about the GPL) for any user-facing software, to force everybody to 'play fair and share'. Th…

MIT license lets you do what you want with the code. That's the deal. The blob storage thing is the real problem though. Nobody talks about it until they try to move their models somewhere else.

Re: The local LLM ecosystem doesn’t need Ollama

#147

Post by audience_mem »

IshKebab wrote: post #112
stuaxo wrote: post #81

[flagged]

Yeah my thoughts exactly. Definitely slop. I have no objection to using AI to help writing. I just don't want to read the same sloppy cliches again and again and again. The short sentences. The Bigger Picture. Here's the rub. It's not just A, it's B. It's like those cliche titles - for fun and profit, the unreasonable effectiveness of, all you need is, etc. etc. but throughout the prose. Stop it guys!

Can you share some excerpts from that article that feel LLM-written to you?

Re: The local LLM ecosystem doesn’t need Ollama

#149

Post by speedgoose »

ekianjo wrote: post #137
speedgoose wrote: post #12

I prefer Ollama over the suggested alternatives. I will switch once we have good user experience on simple features. A new model is released on HF or the Ollama registry? One `ollama pull` and it's available. It's underwhelming? `ollama rm`.

You have no idea what you are downloading with such a pull. At least LMstudio gives you access to all the different versions of the same model.

https://ollama.com/library/gemma4/tags

I see quite a few versions, and I can also use hugging face models.

Re: The local LLM ecosystem doesn’t need Ollama

#150

Post by WobblyDev »

u1hcw9nx wrote: post #118

Two Views of MIT-Style Licenses: 1. MIT-style licenses are "do what you want" as long as you provide a single line of attribution. Including building big closed source business around it. 2. MIT-style licenses are "do what you want" under the law, but they carry moral, GPL-like obligations to think about the "community." To my knowledge Georgi Gerganov, the creator of llama.cpp, has only complained about attribution…

Georgi could have switched to GPL whenever he wanted. He didn't. That's the answer. The loudest voices here aren't the ones writing the code. Meanwhile both projects kept shipping and users got more options. Hard to see the harm.
Post reply on HN