Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The local LLM ecosystem doesn’t need Ollama

sleepingrobots.com

Post reply on HN
Print view View original

201–210 of 222 posts

Re: The local LLM ecosystem doesn’t need Ollama

#201

Post by [deleted] »

JKCalhoun wrote: post #171
cientifico wrote: post #25

For most users that wanted to run LLM locally, ollama solved the UX problem. One command, and you are running the models even with the rocm drivers without knowing. If llama provides such UX, they failed terrible at communicating that. Starting with the name. Llama.cpp: that's a cpp library! Ollama is the wrapper. That's the mental model. I don't want to build my own program! I just want to have fun :-P

"LM Studio… Jan… Msty… koboldcpp…" Plenty of alternatives listed. Can anyone with experience suggest the likely successor to Ollama? I have a Mac Mini but don't mind a C/L tool. I think, as was pointed out, Ollama won because of how easy it is to set up, pull down new models. I would expect similar for a replacement.

[deleted]

Re: The local LLM ecosystem doesn’t need Ollama

#202

Post by maybewhenthesun »

duskdozer wrote: post #153
WobblyDev wrote: post #146

Earlier quoted context omitted.

MIT license lets you do what you want with the code. That's the deal. The blob storage thing is the real problem though. Nobody talks about it until they try to move their models somewhere else.

Well, yeah, which is why it's silly when people use MIT licenses and then complain that those, for example, with the motto "Build > ask. Disrupt or die.", only take and don't contribute anything back, instead of using a license that demands it.

exactly

Re: The local LLM ecosystem doesn’t need Ollama

#203

Post by Zetaphor »

JKCalhoun wrote: post #171
cientifico wrote: post #25

For most users that wanted to run LLM locally, ollama solved the UX problem. One command, and you are running the models even with the rocm drivers without knowing. If llama provides such UX, they failed terrible at communicating that. Starting with the name. Llama.cpp: that's a cpp library! Ollama is the wrapper. That's the mental model. I don't want to build my own program! I just want to have fun :-P

"LM Studio… Jan… Msty… koboldcpp…" Plenty of alternatives listed. Can anyone with experience suggest the likely successor to Ollama? I have a Mac Mini but don't mind a C/L tool. I think, as was pointed out, Ollama won because of how easy it is to set up, pull down new models. I would expect similar for a replacement.

If you don't want to have to think about it, LM Studio is probably the best choice.

Re: The local LLM ecosystem doesn’t need Ollama

#204

Post by Zetaphor »

zozbot234 wrote: post #70
well_ackshually wrote: post #69

Earlier quoted context omitted.

>solved the UX problem. >One command Notwithstanding the fact that there's about zero difference between `ollama run model-name` and `llama-cpp -hf model-name`, and that running things in the terminal is already a gigantic UX blocker (Ollama's popularity comes from the fact that it has a GUI), why are you putting the blame back on an open source project that owes you approximately zero communication ?

> Ollama's popularity comes from the fact that it has a GUI It's not the GUI, it's the curated model hosting platform. Way easier to use than HF for casual users.

LM Studio also offers curation, while giving credit to llama.cpp and also easy search across all of Huggingface's GGUF's

Re: The local LLM ecosystem doesn’t need Ollama

#205

Post by Zetaphor »

shantnutiwari wrote: post #157
mentalgear wrote: post #35

Earlier quoted context omitted.

The reported alternatives seem pretty User-Friendly to me: > LM Studio gives you a GUI if that’s what you want. It uses llama.cpp under the hood, exposes all the knobs, and supports any GGUF model without lock-in. > Jan( https://www.jan.ai/ ) is another open-source desktop app with a clean chat interface and local-first design. > Msty( https://msty.ai/ ) offers a polished GUI with multi-model support and built-in RAG…

All of those options were either too slow, or didnt work for me (Mac with Intel). I could have spent hours googling, but I downloaded Ollama and it just worked. So no, they are not alternatives to ollama

LM Studio is basically Ollama except they give attribution. It offers all of the same features including the ability to host a server.

Re: The local LLM ecosystem doesn’t need Ollama

#207

Post by Zetaphor »

endymion-light wrote: post #106

I'm sorry, on a mac, Ollama just works. It lets me use a model and test it quickly. This is like saying stop using google drive, upload everything to s3 instead! When i'm using Ollama - I honeslty don't care about performance, I'm looking to try out a model and then if it seems good, place it onto a most dedicated stack specifically for it.

Give LM Studio a shot! It gives you the same experience without all of the problems of Ollama.

Re: The local LLM ecosystem doesn’t need Ollama

#208

Post by Zetaphor »

dhruv3006 wrote: post #40

ollama is pretty intuitive to use still - dont see why will stop.

LM Studio is equally as simple, has all the same features, and none of the performance or lock-in problems of ollama.

If you only needed a single reason, how about kneecapping your performance by choosing ollama?

Re: The local LLM ecosystem doesn’t need Ollama

#209

Post by Zetaphor »

IshKebab wrote: post #160
audience_mem wrote: post #147

Earlier quoted context omitted.

Can you share some excerpts from that article that feel LLM-written to you?

Sure. Short sentences like "It shouldn’t be.", "I’ve moved on.", "Ollama didn’t.", etc. Not-this-but-that like "The local LLM ecosystem doesn’t need Ollama. It needs llama.cpp." Weird signposting: "Benchmarks tell the story." Heres-the-rub conclusion: "The Bigger Picture" Starting every title with "The ...". It's definitely largely human-written, but there are enough slop-isms to make it annoying to read. And of cour…

I guess I write like an LLM :P

Probably a side effect of using them so much

Re: The local LLM ecosystem doesn’t need Ollama

#210

Post by thot_experiment »

thot_experiment wrote: post #55

I was pretty big on ollama, it seemed like a great default solution. I had alpha that it was a trash organization but I didn't listen because I just liked having a reliable inference backend that didn't require me to install torch. I switched to llama.cpp for everything maybe 6 months ago because of how fucking frustrating every one of my interactions with ollama (the organization) were. I wanna publicly apologize to…

Oh I was completely wrong about the model management stuff btw, llama-server has fully fledged model management baked in now, you just have to make an *.ini with your model configs (most models can do this themselves, I pointed qwen3.6 at the relevant part of the docs and it wrote me an ini with all of my model configs in about 2 minutes) and you can swap between models via api or a dropdown menu in the UI.
Post reply on HN