Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ollama Turbo

ollama.com

Post reply on HN
Print view View original

111–120 of 251 posts

Re: Ollama Turbo

#111

Post by rpdillon »

mrcwinn wrote: post #105
dcreater wrote: post #94

Called it. It's very unfortunate that the local inference community has aggregated around Ollama when it's clear that's not their long term priority or strategy. Its imperative we move away ASAP

Yes, better to get free sh*t unsustainably. By the way, you're free to create an open source alternative and pour your time into that so we can all benefit. But when you don't — remember I called it!

What? The obvious move is to never have switched to Ollama and just use Llama.cpp directly, which I've been doing for years. Llama.cpp was created first, is the foundation for this product, and is actually open source.

Re: Ollama Turbo

#112

Post by janalsncm »

captainregex wrote: post #109

I am so so so confused as to why Ollama of all companies did this other than an emblematic stab at making money-perhaps to appease someone putting pressure on them to do so. Their stuff does a wonderful job of enabling local for those who want it. So many things to explore there but instead they stand up yet another cloud thing? Love Ollama and hope it stays awesome

The problem is that OSS is free to use but it is not free to create or maintain. If you want it to remain free to use and also up to date, Ollama will need someone to address issues on GitHub. Usually people want to be paid money for that.

Re: Ollama Turbo

#113

Post by A4ET8a8uTh0_v2 »

tarruda wrote: post #106
dcreater wrote: post #94

Called it. It's very unfortunate that the local inference community has aggregated around Ollama when it's clear that's not their long term priority or strategy. Its imperative we move away ASAP

Llama.cpp (library which ollama uses under the hoods) has its own server, and it is fully compatible with open-webui. I moved away from ollama in favor of llama-server a couple of months ago and never missed anything, since I'm still using the same UI.

Interesting, admittedly, I am slowly getting to the point, where ollama's defaults get a little restrictive. If the setup is not too onerous, I would not mind trying. Where did you start?

Re: Ollama Turbo

#114

Post by mchiang »

tarruda wrote: post #106
dcreater wrote: post #94

Called it. It's very unfortunate that the local inference community has aggregated around Ollama when it's clear that's not their long term priority or strategy. Its imperative we move away ASAP

Llama.cpp (library which ollama uses under the hoods) has its own server, and it is fully compatible with open-webui. I moved away from ollama in favor of llama-server a couple of months ago and never missed anything, since I'm still using the same UI.

totally respect your choice, and it's a great project too. Of course as a maintainer of Ollama, my preference is to win you over with Ollama. If it doesn't meet your needs, it's okay. We are more energized than ever to keep improving Ollama. Hopefully one day we will win you back.

Ollama does not use llama.cpp anymore; we do still keep it and occasionally update it to remain compatible for older models for when we used it. The team is great, we just have features we want to build, and want to implement the models directly in Ollama. (We do use GGML and ask partners to help it. This is a project that also powers llama.cpp and is maintained by that same team)

Re: Ollama Turbo

#115

Post by captainregex »

janalsncm wrote: post #112
captainregex wrote: post #109

I am so so so confused as to why Ollama of all companies did this other than an emblematic stab at making money-perhaps to appease someone putting pressure on them to do so. Their stuff does a wonderful job of enabling local for those who want it. So many things to explore there but instead they stand up yet another cloud thing? Love Ollama and hope it stays awesome

The problem is that OSS is free to use but it is not free to create or maintain. If you want it to remain free to use and also up to date, Ollama will need someone to address issues on GitHub. Usually people want to be paid money for that.

money is great! I like money! but if this is their version of buy me a coffee I think there’s room to run elsewhere for their skillset/area of expertise

Re: Ollama Turbo

#116

Post by janalsncm »

gabagool wrote: post #107
Geezus_42 wrote: post #98

Earlier quoted context omitted.

Yeah, the NAZI sex not will be great for business!

You are thinking of Elon Grok, not Groq

When Grok originally came out I thought it was unlucky on Groq’s part. Now that Grok has certain connotations, it’s even more true.

Re: Ollama Turbo

#117

Post by tarruda »

mchiang wrote: post #114
tarruda wrote: post #106

Earlier quoted context omitted.

Llama.cpp (library which ollama uses under the hoods) has its own server, and it is fully compatible with open-webui. I moved away from ollama in favor of llama-server a couple of months ago and never missed anything, since I'm still using the same UI.

totally respect your choice, and it's a great project too. Of course as a maintainer of Ollama, my preference is to win you over with Ollama. If it doesn't meet your needs, it's okay. We are more energized than ever to keep improving Ollama. Hopefully one day we will win you back. Ollama does not use llama.cpp anymore; we do still keep it and occasionally update it to remain compatible for older models for when we us…

> Ollama does not use llama.cpp anymore

That is interesting, did Ollama develop its own proprietary inference engine or did you move to something else?

Any specific reason why you moved away from llama.cpp?

Re: Ollama Turbo

#118

Post by tarruda »

A4ET8a8uTh0_v2 wrote: post #113
tarruda wrote: post #106

Earlier quoted context omitted.

Llama.cpp (library which ollama uses under the hoods) has its own server, and it is fully compatible with open-webui. I moved away from ollama in favor of llama-server a couple of months ago and never missed anything, since I'm still using the same UI.

Interesting, admittedly, I am slowly getting to the point, where ollama's defaults get a little restrictive. If the setup is not too onerous, I would not mind trying. Where did you start?

Download llama-server from llama.cpp Github and install it some PATH directory. AFAIK they don't have an automated installer, so that can be intimidating to some people

Assuming you have llama-server installed, you can download + run a hugging face model with something like

    llama-server -hf ggml-org/gpt-oss-20b-GGUF -c 0 -fa --jinja
And access http://localhost:8080

Re: Ollama Turbo

#119

Post by liuliu »

pagekicker wrote: post #83
liuliu wrote: post #33

Any more information on "Privacy first"? It seems pretty thin if just not retaining data. For Draw Things provided "Cloud Compute", we don't retain any data too (everything is done in RAM per request). But that is still unsatisfactory personally. We will soon add "privacy pass" support, but still not to the satisfactory. Transparency log that can be attested on the hardware would be nice (since we run our open-source…

I see no privacy advantage to working with Ollama, which can sell your data or have it subpoenaed just like anyone else.

In theory, "privacy pass" should help, as you can subpoena content, but cannot know who made these. But that is still thin (and Ollama not doing that too anyway).

Re: Ollama Turbo

#120

Post by mchiang »

tarruda wrote: post #117
mchiang wrote: post #114

Earlier quoted context omitted.

totally respect your choice, and it's a great project too. Of course as a maintainer of Ollama, my preference is to win you over with Ollama. If it doesn't meet your needs, it's okay. We are more energized than ever to keep improving Ollama. Hopefully one day we will win you back. Ollama does not use llama.cpp anymore; we do still keep it and occasionally update it to remain compatible for older models for when we us…

> Ollama does not use llama.cpp anymore That is interesting, did Ollama develop its own proprietary inference engine or did you move to something else? Any specific reason why you moved away from llama.cpp?

it's all open, and specifically, the new models are implemented here: https://github.com/ollama/ollama/tree/main/model/models
Post reply on HN