Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ollama Turbo

ollama.com

Post reply on HN
Print view View original

121–130 of 251 posts

Re: Ollama Turbo

#121

Post by mchiang »

captainregex wrote: post #115
janalsncm wrote: post #112

Earlier quoted context omitted.

The problem is that OSS is free to use but it is not free to create or maintain. If you want it to remain free to use and also up to date, Ollama will need someone to address issues on GitHub. Usually people want to be paid money for that.

money is great! I like money! but if this is their version of buy me a coffee I think there’s room to run elsewhere for their skillset/area of expertise

hmm, I don't think so. This is more of, we want to keep improving Ollama so we can have a great core.

For the users who want GPUs, which cost us money, we will charge money for it. Completely optional.

Re: Ollama Turbo

#122

Post by dawnerd »

daft_pink wrote: post #73
jacekm wrote: post #63

What could be the benefit of paying $20 to Ollama to run inferior models instead of paying the same amount of money to e.g. OpenAI for access to sota models?

I feel the primary benefit of this Ollama Turbo is that you can quickly test and run different models in the cloud that you could run locally if you had the correct hardware. This allows you to try out some open models and better assess if you could buy a dgx box or Mac Studio with a lot of unified memory and build out what you want to do locally without actually investing in very expensive hardware. Certain applicat…

Quickly test… the two models they support? This is just another subscription to quantized models.

Re: Ollama Turbo

#124

Post by ahmedhawas123 »

So much that is interesting about this

For one of the top local open model inference engines of choice - only supporting OSS out of the gate feels like an angle to just ride the hype knowing OSS is announced today "oh OSS came out and you can use Ollama Turbo to use it"

The subscription based pricing is really interesting. Other players offer this but not for API type services. I always imagine that there will be a real pricing war with LLMs with time / as capabilities mature, and going monthly pricing on API services is possibly a symptom of that

What does this mean for the local inference engine? Does Ollama have enough resources to maintain both?

Re: Ollama Turbo

#125

Post by iphone_elegance »

rafram wrote: post #14
mrs6969 wrote: post #9

Earlier quoted context omitted.

Agreed. Ollama itself is kind a wrapper around llamacpp anyway. Feel like the real guy is not included to the process. Now I am going to go and write a wrapper around llamacpp, that is only open source, truly local. How can I trust ollama to not to sell my data.

Ollama is not a wrapper around llama.cpp anymore, at least for multimodal models (not sure about others). They have their own engine: https://ollama.com/blog/multimodal-models

looks like the backend is ggml, am I missing something? same diff

Re: Ollama Turbo

#126

Post by irthomasthomas »

mchiang wrote: post #104
irthomasthomas wrote: post #101

If these are FP4 like the other ollama models then I'm not very interested. If I'm using an API anyway I'd rather use the full weights.

OpenAI has only provided MXFP4 weights. These are the same weights used by other cloud providers.

Oh, I didn't know that. Weird!

Re: Ollama Turbo

#127

Post by reissbaker »

irthomasthomas wrote: post #126
mchiang wrote: post #104

Earlier quoted context omitted.

OpenAI has only provided MXFP4 weights. These are the same weights used by other cloud providers.

Oh, I didn't know that. Weird!

It was natively trained in FP4. Probably both to reduce VRAM usage at inference time (fits on a single H100), and to allow better utilization of B200s (which are especially fast for FP4).

Re: Ollama Turbo

#128

Post by pogue »

liuliu wrote: post #33

Any more information on "Privacy first"? It seems pretty thin if just not retaining data. For Draw Things provided "Cloud Compute", we don't retain any data too (everything is done in RAM per request). But that is still unsatisfactory personally. We will soon add "privacy pass" support, but still not to the satisfactory. Transparency log that can be attested on the hardware would be nice (since we run our open-source…

I would pay more if they let you run the models in Switzerland or some other GDPR respecting country, even if there was extra latency. I would also hope everything is being sent over SSL or something similar.

Re: Ollama Turbo

#129

Post by seanmcdirmid »

pogue wrote: post #128
liuliu wrote: post #33

Any more information on "Privacy first"? It seems pretty thin if just not retaining data. For Draw Things provided "Cloud Compute", we don't retain any data too (everything is done in RAM per request). But that is still unsatisfactory personally. We will soon add "privacy pass" support, but still not to the satisfactory. Transparency log that can be attested on the hardware would be nice (since we run our open-source…

I would pay more if they let you run the models in Switzerland or some other GDPR respecting country, even if there was extra latency. I would also hope everything is being sent over SSL or something similar.

I had to do a double take here. Switzerland surely isn’t in the GDPR, so you mean their own privacy laws or GDPR in the EU?

Re: Ollama Turbo

#130

Post by om8 »

tarruda wrote: post #106
dcreater wrote: post #94

Called it. It's very unfortunate that the local inference community has aggregated around Ollama when it's clear that's not their long term priority or strategy. Its imperative we move away ASAP

Llama.cpp (library which ollama uses under the hoods) has its own server, and it is fully compatible with open-webui. I moved away from ollama in favor of llama-server a couple of months ago and never missed anything, since I'm still using the same UI.

It’s unfortunate that llama.cpp’s code is a mess. It’s impossible to make any meaningful contributions to it.
Post reply on HN