Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ollama Turbo

ollama.com

Post reply on HN
Print view View original

141–150 of 251 posts

Re: Ollama Turbo

#141

Post by kristjansson »

mchiang wrote: post #114
tarruda wrote: post #106

Earlier quoted context omitted.

Llama.cpp (library which ollama uses under the hoods) has its own server, and it is fully compatible with open-webui. I moved away from ollama in favor of llama-server a couple of months ago and never missed anything, since I'm still using the same UI.

totally respect your choice, and it's a great project too. Of course as a maintainer of Ollama, my preference is to win you over with Ollama. If it doesn't meet your needs, it's okay. We are more energized than ever to keep improving Ollama. Hopefully one day we will win you back. Ollama does not use llama.cpp anymore; we do still keep it and occasionally update it to remain compatible for older models for when we us…

> Ollama does not use llama.cpp anymore;

> We do use GGML

Sorry, but this is kind of hiding the ball. You don't use llama.cpp, you just ... use their core library that implements all the difficult bits, and carry a patchset on top of it?

Why do you have to start with the first statement at all? "we use the core library from llama.cpp/ggml and implement what we think is a better interface and UX. we hope you like it and find it useful."

Re: Ollama Turbo

#142

Post by fluidcruft »

daft_pink wrote: post #73
jacekm wrote: post #63

What could be the benefit of paying $20 to Ollama to run inferior models instead of paying the same amount of money to e.g. OpenAI for access to sota models?

I feel the primary benefit of this Ollama Turbo is that you can quickly test and run different models in the cloud that you could run locally if you had the correct hardware. This allows you to try out some open models and better assess if you could buy a dgx box or Mac Studio with a lot of unified memory and build out what you want to do locally without actually investing in very expensive hardware. Certain applicat…

Me at home: $20/mo while I wait for a card that can run this or dgx box? Decisions, decisions.

Re: Ollama Turbo

#143

Post by cchance »

dcreater wrote: post #94

Called it. It's very unfortunate that the local inference community has aggregated around Ollama when it's clear that's not their long term priority or strategy. Its imperative we move away ASAP

I stopped using them when they started doing the weird model naming bullshit stuck with lmstudio since

Re: Ollama Turbo

#144

Post by tomrod »

mchiang wrote: post #99
dcreater wrote: post #94

Called it. It's very unfortunate that the local inference community has aggregated around Ollama when it's clear that's not their long term priority or strategy. Its imperative we move away ASAP

hmm, how so? Ollama is open and the pricing is completely optional for users who want additional GPUs. Is it bad to fairly charge money for selling GPUs that cost us money too, and use that money to grow the core open-source project? At one point, it just has to be reasonable. I'd like to believe by having a conscientious, we can create something great.

Everyone just wants to solarpunk this up.

Re: Ollama Turbo

#146

Post by selcuka »

Havoc wrote: post #136

That'll be an uphill battle on value proposition tbh. $20 a month for access to a widely available MoE 120B with ~5B active parameters at unspecified usage limits? I guess their target audience values convenience and easy of use above all else so that could play well there maybe.

> Turbo includes hourly and daily limits to avoid capacity issues. Usage-based pricing will soon be available to consume models in a metered fashion.

Doesn't look that much better than a ChatGPT Plus subscription.

Re: Ollama Turbo

#148

Post by Aurornis »

dcreater wrote: post #94

Called it. It's very unfortunate that the local inference community has aggregated around Ollama when it's clear that's not their long term priority or strategy. Its imperative we move away ASAP

> Its imperative we move away ASAP

Why? If the tool works then use it. They’re not forcing you to use the cloud.

Re: Ollama Turbo

#149

Post by agnishom »

> What is Turbo?

> Turbo is a new way to run open models using datacenter-grade hardware.

What? Why not just say that it is a cloud-based service for running models? Why this language?

Re: Ollama Turbo

#150

Post by jmort »

liuliu wrote: post #33

Any more information on "Privacy first"? It seems pretty thin if just not retaining data. For Draw Things provided "Cloud Compute", we don't retain any data too (everything is done in RAM per request). But that is still unsatisfactory personally. We will soon add "privacy pass" support, but still not to the satisfactory. Transparency log that can be attested on the hardware would be nice (since we run our open-source…

I don't see a privacy policy and their desktop app is closed source. So, not encouraging.

[full disclosure I am working on something with actual privacy guarantees for LLM calls that does use a transparency log, etc.]

Post reply on HN