The local LLM ecosystem doesn’t need Ollama
sleepingrobots.com
The local LLM ecosystem doesn’t need Ollama
1–10 of 222 posts
Re: The local LLM ecosystem doesn’t need Ollama
#2I got tired of repeating the same points and having to dig up sources every time, so here's the timeline (as I know it) in one place with sources.
Re: The local LLM ecosystem doesn’t need Ollama
#3Do they still not let you change the default model folder? You had to go through this whole song and dance to manually register a model via a pointless dockerfile wannabe that then seemed to copy the original model into their hash storage (again, unable to change where that storage lived).
At the time I dropped it for LMStudio, which to be fair was not fully open source either, but at least exposed the model folder and integrated with HF rather than a proprietary model garden for no good reason.
Re: The local LLM ecosystem doesn’t need Ollama
#4On a practical note if fumbles connection handling as to be unusable to download anything.
Re: The local LLM ecosystem doesn’t need Ollama
#5Do they still not let you change the default model folder? You had to go through this whole song and dance to manually register a model via a pointless dockerfile wannabe that then seemed to copy the original model into their hash storage (again, unable to change where that storage lived). At the time I dropped it for LMStudio, which to be fair was not fully open source either, but at least exposed the model folder a…
This also annoyed me a lot. I was running it before upgrading the SSD storage and I wanted to compare with LM Studio. Figured it would be good to have both interfaces use the same models downloaded from HF.
Had to go down the same rabbit hole of finding where things are, how they're sorted/separated/etc. It was unnecessarily painful
Re: The local LLM ecosystem doesn’t need Ollama
#6I think the biggest advantage for me with ollama is the ability to "hotswap" models with different utility instead of restarting the server with different models combined with the simple "ollama pull model". In other words, it has been quite convenient.
Due to this post I had to search a bit and it seems that llama.cpp recently got router support[1], so I need to have a look at this.
My main use for this is a discord bot where I have different models for different features like replying to messages with images/video or pure text, and non reply generation of sentiment and image descriptions. These all perform best with different models and it has been very convenient for the server to just swap in and out models on request.
[1] https://huggingface.co/blog/ggml-org/model-management-in-lla...
Re: The local LLM ecosystem doesn’t need Ollama
#7I think the biggest advantage for me with ollama is the ability to "hotswap" models with different utility instead of restarting the server with different models combined with the simple "ollama pull model". In other words, it has been quite convenient. Due to this post I had to search a bit and it seems that llama.cpp recently got router support[1], so I need to have a look at this. My main use for this is a discord…
You can do that with llama-server
Re: The local LLM ecosystem doesn’t need Ollama
#8I got tired of repeating the same points and having to dig up sources every time, so here's the timeline (as I know it) in one place with sources.
Thanks, did not know any of this.
Re: The local LLM ecosystem doesn’t need Ollama
#9i use goose by block
Re: The local LLM ecosystem doesn’t need Ollama
#10I got tired of repeating the same points and having to dig up sources every time, so here's the timeline (as I know it) in one place with sources.
Great writing, thanks for the summary and timeline.