Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The local LLM ecosystem doesn’t need Ollama

sleepingrobots.com

Post reply on HN
Print view View original

131–140 of 222 posts

Re: The local LLM ecosystem doesn’t need Ollama

#131

Post by pplonski86 »

I like Ollama Cloud service (I'm paid pro user), because it let me test several open source LLMs very fast - I dont need to download anything locally, just change the model name in the API. If I like the model then I can download it and run locally with sensitive data. I also like their CLI, because it is simple to use.

The fact that they are trying to make money is normal - they are a company. They need to pay the bills.

I agree that they should improve communication, but I assume it is still small company with a lot of different requests, and some things might be overlooked.

Overall I like the software and services they provide.

Re: The local LLM ecosystem doesn’t need Ollama

#133

Post by wrxd »

song wrote: post #99

So, on a mac, what good alternative to ollama supports mlx for acceleration? My main use case is that I have an old m1 max macbook pro with 64 gb ram that I use as a model server.

I read good things about https://omlx.ai but I don't really know enough of the ecosystem to know if there are better options.

If someone has opinions please let us know!

Re: The local LLM ecosystem doesn’t need Ollama

#134

Post by tasuki »

asim wrote: post #88

Ah man the VC death trap. It's ok. I don't mean it like that but this is classic. It's unavoidable. They gotta make money. They took money, they gotta make money. It's not easy. Everyone has principles, developers more than anyone. They are developers, they are people like you and me. They didn't even start as ollama. They started as a kubernetes infra project in YC and pivoted. Listen don't be hard on these guys. It…

> This stuff is messy. Give them some credit. They give you an epic open source project.

It seems to me the epic open source project was given to us by Georgi Gerganov. These people just tried to milk it for some money, and made everything a little worse in the process.

Re: The local LLM ecosystem doesn’t need Ollama

#135

Post by ekianjo »

zozbot234 wrote: post #60
usernomdeguerre wrote: post #3

Do they still not let you change the default model folder? You had to go through this whole song and dance to manually register a model via a pointless dockerfile wannabe that then seemed to copy the original model into their hash storage (again, unable to change where that storage lived). At the time I dropped it for LMStudio, which to be fair was not fully open source either, but at least exposed the model folder a…

> Do they still not let you change the default model folder? Actually they do. It's environment variable OLLAMA_MODELS in the server configuration file.

Yet people claim it has great UI... And still you can't define that in their GUI...

Re: The local LLM ecosystem doesn’t need Ollama

#136

Post by kgeist »

0xbadcafebee wrote: post #13

No mention of the fact that Ollama is about 1000x easier to use. Llama.cpp is a great project, but it's also one of the least user friendly pieces of software I've used. I don't think anyone in the project cares about normal users. I started with Ollama, and it was great. But I moved to llama.cpp to have more up-to-date fixes. I still use Ollama to pull and list my models because it's so easy. I then built my own set…

>No mention of the fact that Ollama is about 1000x easier to use

I remember changing the context size from the default unusable 2k to something bigger the model actually supports required creating a new model file in Ollama if you wanted the change to persist (another alternative: set an env var before running ollama; although, if you go that low-level route, why not just launch llama.cpp). How was that easier? Did they change this?

I remember people complaining model X is "dumb" simply because Ollama capped the context size to a ridiculously small number by default.

IMHO trying to model Ollama after Docker actually makes it harder for casual users. And power users will have it easier with llama.cpp directly

Re: The local LLM ecosystem doesn’t need Ollama

#137

Post by ekianjo »

speedgoose wrote: post #12

I prefer Ollama over the suggested alternatives. I will switch once we have good user experience on simple features. A new model is released on HF or the Ollama registry? One `ollama pull` and it's available. It's underwhelming? `ollama rm`.

You have no idea what you are downloading with such a pull. At least LMstudio gives you access to all the different versions of the same model.

Re: The local LLM ecosystem doesn’t need Ollama

#138

Post by rowendduke »

0xbadcafebee wrote: post #13

No mention of the fact that Ollama is about 1000x easier to use. Llama.cpp is a great project, but it's also one of the least user friendly pieces of software I've used. I don't think anyone in the project cares about normal users. I started with Ollama, and it was great. But I moved to llama.cpp to have more up-to-date fixes. I still use Ollama to pull and list my models because it's so easy. I then built my own set…

Not like it mattered much to me but llama-cpp is way lighter and 10x smaller in size.

Resumable downloads seem to work better in llama-cpp.

I love the inbuilt GUI.

I used ollama first and honestly, llama-cpp has been a much better experience.

Maybe given enough time, I would have seen the benefit of ollama but the inability to turn off updates even after users requested it extensively made me uninstall it. Postman PTSD is real.

Re: The local LLM ecosystem doesn’t need Ollama

#139

Post by dizhn »

> the file gets copied into Ollama’s hashed blob storage, you still can’t share the GGUF with other tool

This is the reason I had stopped using it. I think they might be doing it for deduplication however it makes it impossible to use the same model with other tools. Every other tool can just point to the same existing gguf and can go. Whether its their intention or not, it's making it difficult to try out other tools. Model files are quite large as you know and storage and download can become issues. (They are for me)

Re: The local LLM ecosystem doesn’t need Ollama

#140

Post by carlostkd »

tosh wrote: post #84

This is a bit like saying stop using Ubuntu, use Debian instead. Both llama.cpp and ollama are great and focused on different things and yet complement each other (both can be true at the same time!) Ollama has great ux and also supports inference via mlx, which has better performance on apple silicon than llama.cpp I'm using llama.cpp, ollama, lm studio, mlx etc etc depending on what is most convenient for me at the…

There isn't much you can do with Ollama models besides saying good morning.
Post reply on HN