Can anyone explain why this is a bad thing?
Is it because they developed s new ollama which isn't open and which doesn't use llama.cpp?
Ollama Turbo
171–180 of 251 posts
Re: Ollama Turbo
#172Called it. It's very unfortunate that the local inference community has aggregated around Ollama when it's clear that's not their long term priority or strategy. Its imperative we move away ASAP
Local inference is becoming completely commoditized imo. These days even docker has a local models you can launch with a single click (or command).
Re: Ollama Turbo
#173Called it. It's very unfortunate that the local inference community has aggregated around Ollama when it's clear that's not their long term priority or strategy. Its imperative we move away ASAP
[deleted]
Re: Ollama Turbo
#174Earlier quoted context omitted.
totally respect your choice, and it's a great project too. Of course as a maintainer of Ollama, my preference is to win you over with Ollama. If it doesn't meet your needs, it's okay. We are more energized than ever to keep improving Ollama. Hopefully one day we will win you back. Ollama does not use llama.cpp anymore; we do still keep it and occasionally update it to remain compatible for older models for when we us…
So I’m using turbo and just want to provide some feedback. I can’t figure out how to connect raycast and project goose to ollama turbo. The software that calls it essentially looks for the models via ollama but cannot find the turbo ones and the documentation is not clear yet. Just my two cents, the inference is very quick and I’m happy with the speed but not quite usable yet.
so sorry about this. We are learning. Possible to email, and we will first make it right while we improve Ollama's turbo mode. hello@ollama.com
Re: Ollama Turbo
#175Earlier quoted context omitted.
But Ollama is a toy, it's meaningful for hobbyists and individuals to use locally like myself. Why would it be the right choice for anything more? AWS, vLLM, SGLang etc would be the solutions for enterprise I knew a startup that deployed ollama on a customers premises and when I asked them why, they had absolutely no good reason. Likely they did it because it was easy. That's not the "easy to use" case you want to so…
Honestly, I think it just depends. A few hours ago I wrote I would never want it for a production setting but actually if I was standing something up myself and I could just download headless ollama and know it would work. Hey, that would also be fine most likely. Maybe later on I'd revisit it from a devops perspective, and refactor deployment methodology/stack, etc. Maybe I'd benchmark it and realize its fine actual…
But its effectively equally easy to do the same with llama.cpp, vllm or modular..
(any differences are small enough that they either shouldn't cause the human much work or can very easily be delegated to AI)
Re: Ollama Turbo
#176Ah, vague "limits". Hard pass.
Re: Ollama Turbo
#177I like how the landing page (and even this HN page until this point) completely miss any reference to Meta and Facebook.
The landing page promises privacy but anyone who knows how FB used VPN software to spy on people, knows that as long as the current leadership is in place, we shouldn't assume they've all of a sudden became fans of our privacy.
Re: Ollama Turbo
#178I like how the landing page (and even this HN page until this point) completely miss any reference to Meta and Facebook. The landing page promises privacy but anyone who knows how FB used VPN software to spy on people, knows that as long as the current leadership is in place, we shouldn't assume they've all of a sudden became fans of our privacy.
Ollama isn’t connected to Meta besides offering Llama as one of the potential models you can run.
There is obviously some connection to Llama (the original models giving rise to llama.cpp which Ollama was built on) but the companies have no affiliation.
Re: Ollama Turbo
#179Called it. It's very unfortunate that the local inference community has aggregated around Ollama when it's clear that's not their long term priority or strategy. Its imperative we move away ASAP
Oh no this is a positively diabolical development, offering...hosting services tailored to a specific use case at a reasonable price ...
They can’t keep getting away with this.
Re: Ollama Turbo
#180The 'Sign In' link on the Ollama Mac App when you click Turbo doesn't work...
It should open ollama.com/connect – sorry about that. Feel free to message me jeff@ollama.com if you keep seeing issues