[flagged]
The local LLM ecosystem doesn’t need Ollama
81–90 of 222 posts
Re: The local LLM ecosystem doesn’t need Ollama
#82It feels like a bit of history is missing... If ollama was founded 3 years before llama.cpp was released, what engine did they use then? When did they transition?
They spent several years in stealth mode but the initial release was llama.cpp. Ollama v0.0.1 "Fast inference server written in Go, powered by llama.cpp" https://github.com/ollama/ollama/tree/v0.0.1
They spent several years in stealth mode
doing what?
trying to build themselves what llama.cpp ended up doing for them?
Re: The local LLM ecosystem doesn’t need Ollama
#83Has anybody figured some of the best flags to compile llama.cpp for rocm? I'm using the framework desktop and the Vulkan backend, because it was easier to compile out of the box, but I feel there's large peformance gains on the table by swtiching to rocm. Not sure if installing with brew on ubuntu would be easier.
Re: The local LLM ecosystem doesn’t need Ollama
#84This is a bit like saying stop using Ubuntu, use Debian instead.
Both llama.cpp and ollama are great and focused on different things and yet complement each other (both can be true at the same time!)
Ollama has great ux and also supports inference via mlx, which has better performance on apple silicon than llama.cpp
I'm using llama.cpp, ollama, lm studio, mlx etc etc depending on what is most convenient for me at the time to get done what I want to get done (e.g. a specific model config to run, mcp, just try a prompt quickly, …)
Re: The local LLM ecosystem doesn’t need Ollama
#85I am trying to run models that are on the edge of what my hardware can support. I guess many people are.
So given, as the author states, Ollama runs the LLMs inefficiently, what is the tool that runs them most efficiently on limited hardware ?
Re: The local LLM ecosystem doesn’t need Ollama
#86This is a bit like saying stop using Ubuntu, use Debian instead. Both llama.cpp and ollama are great and focused on different things and yet complement each other (both can be true at the same time!) Ollama has great ux and also supports inference via mlx, which has better performance on apple silicon than llama.cpp I'm using llama.cpp, ollama, lm studio, mlx etc etc depending on what is most convenient for me at the…
The author points out that the Ollama people are evil.
So it is more like saying "Stop using SCO Unix, use Linux instead".
Re: The local LLM ecosystem doesn’t need Ollama
#87Earlier quoted context omitted.
While that might be true, for as long as its name is “.cpp”, people are going to think it’s a C++ library and avoid it.
This is the first I'm learning that it isn't just a C++ library. In fact the first line of the wikipedia article is: > llama.cpp is an open source software library
[deleted]
Re: The local LLM ecosystem doesn’t need Ollama
#88Ah man the VC death trap. It's ok. I don't mean it like that but this is classic. It's unavoidable. They gotta make money. They took money, they gotta make money. It's not easy. Everyone has principles, developers more than anyone. They are developers, they are people like you and me. They didn't even start as ollama. They started as a kubernetes infra project in YC and pivoted. Listen don't be hard on these guys. It's hard enough. Trust me I did it. And not as well them.
This is the game. We shouldn't delude ourselves into thinking there are alternative ways to become profitable around open source, there aren't. You effectively end up in this trap and there's no escape and then you have to compromise on everything to build the company, return the money, make a profit. You took people's money, now you have to make good, there's no choice. And anyone who thinks differently is deluded. Open source only goes one way. To the enterprise. Everything else is burning money and wasting time. Look at Docker. Textbook example of the enormous struggle to capture the value of a project that had so much potential, defined an industry and ultimately failed. Even the reboot failed. Sorry. It did.
This stuff is messy. Give them some credit. They give you an epic open source project. Be grateful for that. And now if you want to move on, move on. They don't need a hard time. They're already having a hard time. These guys are probably sweating bullets trying to make it work while their investors breathe down their necks waiting for the payoff. Let them breathe.
Good luck to you ollama guys!
Re: The local LLM ecosystem doesn’t need Ollama
#89[flagged]
Re: The local LLM ecosystem doesn’t need Ollama
#90This is a bit like saying stop using Ubuntu, use Debian instead. Both llama.cpp and ollama are great and focused on different things and yet complement each other (both can be true at the same time!) Ollama has great ux and also supports inference via mlx, which has better performance on apple silicon than llama.cpp I'm using llama.cpp, ollama, lm studio, mlx etc etc depending on what is most convenient for me at the…
> Both llama.cpp and ollama are great and focused on different things and yet complement each other
According to the article, ollama is not great (that’s an understatement), focused on making money for the company, stealing clout and nothing else, and hardly complements llama.cpp at all since not long after the initial launch. All of these are backed by evidence.
You may disagree, but then you need to refute OP’s points, not try to handwave them away with a BS analogy that’s nothing like the original.