Called it. It's very unfortunate that the local inference community has aggregated around Ollama when it's clear that's not their long term priority or strategy. Its imperative we move away ASAP
happy sglang user here :)
Ollama Turbo
151–160 of 251 posts
Re: Ollama Turbo
#152What could be the benefit of paying $20 to Ollama to run inferior models instead of paying the same amount of money to e.g. OpenAI for access to sota models?
I feel the primary benefit of this Ollama Turbo is that you can quickly test and run different models in the cloud that you could run locally if you had the correct hardware. This allows you to try out some open models and better assess if you could buy a dgx box or Mac Studio with a lot of unified memory and build out what you want to do locally without actually investing in very expensive hardware. Certain applicat…
> quickly test and run different models in the cloud that you could run locally if you had the correct hardware.
I feel like they're competing against Hugging Face or even Colaboratory then if this is the case.
And for cases that require strict privacy control, I don't think I'd run it on emergent models or if I really have to, I would prefer doing so on an existing cloud setup already that has the necessary trust / compliance barriers addressed. (does Ollama Turbo even have their Trust center up?)
I can see its potential once it gets rolling, since there's a lot of ollama installations out there.
Re: Ollama Turbo
#153Nice release. Part of the problem right now with OSS models (at least for enterprise users) is the diversity of offerings in terms of: - Speed - Cost - Reliability - Feature Parity (eg: context caching) - Performance (What quant level is being used...really?) - Host region/data privacy guarantees - LTS And that's not even including the decision of what model you want to use! Realistically if you want to use an OSS mo…
true but ignores handing over all your prompt traffic without any real legal protections as sama has pointed out: [1] https://californiarecorder.com/sam-altman-requires-ai-privil...
I wouldn't be surprised if those undeleted chats or some inferred data that is based on it is part of the gpt-5 training data. Somehow I don't trust this sama guy at all.
Re: Ollama Turbo
#154Earlier quoted context omitted.
totally respect your choice, and it's a great project too. Of course as a maintainer of Ollama, my preference is to win you over with Ollama. If it doesn't meet your needs, it's okay. We are more energized than ever to keep improving Ollama. Hopefully one day we will win you back. Ollama does not use llama.cpp anymore; we do still keep it and occasionally update it to remain compatible for older models for when we us…
> Ollama does not use llama.cpp anymore; > We do use GGML Sorry, but this is kind of hiding the ball. You don't use llama.cpp, you just ... use their core library that implements all the difficult bits, and carry a patchset on top of it? Why do you have to start with the first statement at all? "we use the core library from llama.cpp/ggml and implement what we think is a better interface and UX. we hope you like it a…
thanks, I'll take that feedback, but I do want to clarify that it's not from llama.cpp/ggml. It's from ggml-org/ggml. I supposed it's all interchangeable though, so thank you for it.
Re: Ollama Turbo
#155Why does everything AI-related have to be $20? Why can't there be tiers? OpenAI setting the standard of $20/m for every AI application is one of the worst things to ever happen.
https://openai.com/chatgpt/pricing/ - $0 / $20 / $200 / $25 (team) / custom enterprise pricing / on-demand API pricing https://www.anthropic.com/pricing - $0 / $17 (if billed annually) / $20 (if billed monthly) / $100 / $25 (team) / custom enterprise pricing / on-demand API pricing Sounds like tiers to me.
I should have specified less expensive tiers (below the $20 standard). A tier <= $10 would be great. Anything over $10 for casual use seems excessive (or at least from my perspective)
Re: Ollama Turbo
#156No thanks, Ollama. I'd rather give the money to anyone but you grifters.
Re: Ollama Turbo
#157Earlier quoted context omitted.
Llama.cpp (library which ollama uses under the hoods) has its own server, and it is fully compatible with open-webui. I moved away from ollama in favor of llama-server a couple of months ago and never missed anything, since I'm still using the same UI.
totally respect your choice, and it's a great project too. Of course as a maintainer of Ollama, my preference is to win you over with Ollama. If it doesn't meet your needs, it's okay. We are more energized than ever to keep improving Ollama. Hopefully one day we will win you back. Ollama does not use llama.cpp anymore; we do still keep it and occasionally update it to remain compatible for older models for when we us…
I’ve never seen a PR on ggml from Ollama folks though. Could you mention one contribution you did?
Re: Ollama Turbo
#158Earlier quoted context omitted.
Llama.cpp (library which ollama uses under the hoods) has its own server, and it is fully compatible with open-webui. I moved away from ollama in favor of llama-server a couple of months ago and never missed anything, since I'm still using the same UI.
totally respect your choice, and it's a great project too. Of course as a maintainer of Ollama, my preference is to win you over with Ollama. If it doesn't meet your needs, it's okay. We are more energized than ever to keep improving Ollama. Hopefully one day we will win you back. Ollama does not use llama.cpp anymore; we do still keep it and occasionally update it to remain compatible for older models for when we us…
So I’m using turbo and just want to provide some feedback. I can’t figure out how to connect raycast and project goose to ollama turbo. The software that calls it essentially looks for the models via ollama but cannot find the turbo ones and the documentation is not clear yet. Just my two cents, the inference is very quick and I’m happy with the speed but not quite usable yet.
Re: Ollama Turbo
#159Ollama is great but I feel like Georgi Gerganov deserves way more credit for llama.cpp. He (almost) single-handedly brought LLMs to the masses. With the latest news of some AI engineers' compensation reaching up to a billion dollars, feels a bit unfair that Georgi is not getting a much larger slice of the pie.
Seriously, people astroturfing this thread by saying ollama has a new engine. It literally is the same engine that llama.cpp uses and georgi and slaren maintain! VC funding will make people so dishonest and just plain grifters
Re: Ollama Turbo
#160Earlier quoted context omitted.
[flagged]
sorry that you feel the way you feel. :( I'm not sure which package we use that is triggering this. My guess is llama.cpp based on what I see on social? Ollama has long shifted to using our own engine. We do use llama.cpp for legacy and backwards compatibility. I want to be clear it's not a knock on the llama.cpp project either. There are certain features we want to build into Ollama, and we want to be opinionated on…
> Have you supported our past gigs before?
Docker Desktop? One of the most memorable private equity rugpulls in developer tooling?
Fool me once shame on you, fool me twice shame on me