Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The local LLM ecosystem doesn’t need Ollama

sleepingrobots.com

Post reply on HN
Print view View original

111–120 of 222 posts

Re: The local LLM ecosystem doesn’t need Ollama

#111

Post by UqWBcuFx6NV4r »

matja wrote: post #101
tosh wrote: post #84

This is a bit like saying stop using Ubuntu, use Debian instead. Both llama.cpp and ollama are great and focused on different things and yet complement each other (both can be true at the same time!) Ollama has great ux and also supports inference via mlx, which has better performance on apple silicon than llama.cpp I'm using llama.cpp, ollama, lm studio, mlx etc etc depending on what is most convenient for me at the…

> This is a bit like saying stop using Ubuntu, use Debian instead. Not really, because Ubuntu has always acknowledged Debian and explicitly documented the dependency: > Debian is the rock on which Ubuntu is built. > Ubuntu builds on the Debian architecture and infrastructure and collaborates widely with Debian developers, but there are important differences. Ubuntu has a distinctive user interface, a separate develop…

OK. That says absolutely nothing about actual UX or anything that matters to most actual users (as opposed to argumentative HN ideologues).

Re: The local LLM ecosystem doesn’t need Ollama

#112

Post by IshKebab »

stuaxo wrote: post #81

[flagged]

Yeah my thoughts exactly. Definitely slop. I have no objection to using AI to help writing. I just don't want to read the same sloppy cliches again and again and again. The short sentences. The Bigger Picture. Here's the rub. It's not just A, it's B.

It's like those cliche titles - for fun and profit, the unreasonable effectiveness of, all you need is, etc. etc. but throughout the prose. Stop it guys!

Re: The local LLM ecosystem doesn’t need Ollama

#114

Post by teekert »

roosgit wrote: post #102
teekert wrote: post #63

Earlier quoted context omitted.

Was hoping it was so easy :) But I probably need to look into it some more. llama_model_load: error loading model: error loading model architecture: unknown model architecture: 'gemma4' llama_model_load_from_file_impl: failed to load model Edit: @below, I used `nix-shell -p llama-cpp` so not brew related. Could indeed be an older version indeed! I'll check.

I just hit that error a few minutes ago. I build my llama.cpp from source because I use CUDA on Linux. So I made the mistake of trying to run Gemma4 on an older version I had and I got the same error. It’s possible brew installs an older version which doens’t support Gemma4 yet.

Ah it was indeed just that!

I'm now on:

$ llama --version version: 8770 (82764d8) built with GNU 15.2.0 for Linux x86_64

(From Nix unstable)

And this works as advertised, nice chat interface, but no openai API I guess, so no opencode...

Re: The local LLM ecosystem doesn’t need Ollama

#115

Post by [deleted] »

tyfon wrote: post #6

I think the biggest advantage for me with ollama is the ability to "hotswap" models with different utility instead of restarting the server with different models combined with the simple "ollama pull model". In other words, it has been quite convenient. Due to this post I had to search a bit and it seems that llama.cpp recently got router support[1], so I need to have a look at this. My main use for this is a discord…

[deleted]

Re: The local LLM ecosystem doesn’t need Ollama

#118

Post by u1hcw9nx »

Two Views of MIT-Style Licenses:

1. MIT-style licenses are "do what you want" as long as you provide a single line of attribution. Including building big closed source business around it.

2. MIT-style licenses are "do what you want" under the law, but they carry moral, GPL-like obligations to think about the "community."

To my knowledge Georgi Gerganov, the creator of llama.cpp, has only complained about attribution when it was missing. As an open-source developer, he selected a permissive license and has not complained about other issues, only the lack of credit. It seems he treats the MIT license as the first kind.

The article has other good points not related to licensing that are good to know. Like performance issues and simplicity that makes me consider llama.cpp.

Re: The local LLM ecosystem doesn’t need Ollama

#120

Post by brabel »

endymion-light wrote: post #106

I'm sorry, on a mac, Ollama just works. It lets me use a model and test it quickly. This is like saying stop using google drive, upload everything to s3 instead! When i'm using Ollama - I honeslty don't care about performance, I'm looking to try out a model and then if it seems good, place it onto a most dedicated stack specifically for it.

Ollama is a bit easier to use, you’re right. But the point of the article is the way they just disregarded the license of llama.cpp, moved away from open source while still claiming to be open source and pivoted to cloud offerings when the whole point was to run local models all while without contributing anything back to the big open source projects it owns its existence to. Maybe you don’t care about performance (weird given performance is the main blocker for local LLMs) but you should care about the ethics of companies making the product you use?

And anyway this thread has lots of alternatives that are even easier to use and don’t shit on the open source community making things happen.

Post reply on HN