Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ollama Turbo

ollama.com

Post reply on HN
Print view View original

221–230 of 251 posts

Re: Ollama Turbo

#221

Post by dcreater »

Hasnep wrote: post #198
dcreater wrote: post #185

Earlier quoted context omitted.

There are many, many FOSS apps that use Ollama as a dependency. If Ollama rugs, then all those projects suffer. Its a tale we seen played out many times. Redis is the most recent example.

Most apps that integrate with ollama that I've seen just have an OpenAI compatible API parameter which defaults to port 11434 which ollama uses, but can be changed easily. Is there a way to integrate ollama more deeply?

Yes, but I fear the average person will not understand that and assume you need Ollama. That false perception is sufficiently damaging im afraid

Re: Ollama Turbo

#222

Post by evilduck »

dcreater wrote: post #175
jnmandal wrote: post #162

Earlier quoted context omitted.

Honestly, I think it just depends. A few hours ago I wrote I would never want it for a production setting but actually if I was standing something up myself and I could just download headless ollama and know it would work. Hey, that would also be fine most likely. Maybe later on I'd revisit it from a devops perspective, and refactor deployment methodology/stack, etc. Maybe I'd benchmark it and realize its fine actual…

But its effectively equally easy to do the same with llama.cpp, vllm or modular.. (any differences are small enough that they either shouldn't cause the human much work or can very easily be delegated to AI)

Llama.cpp is not really that easy unless you're supported by their prebuilt binaries. Go to the llama.cpp GitHub page and find a prebuilt CUDA enabled release for a Fedora based linux distro. Oh there isn't one you say? Welcome to losing an hour or more of your time.

Then you want to swap models on the fly. llama-swap you say? You now get to learn a new custom yaml based config file syntax that does basically nothing that the Ollama model file already does so that you can ultimately... have the same experience as Ollama but now you've lost hours just to get back to square one.

Then you need it to start and be ready with the system reboot? Great, now you get to write some systemd services, move stuff into system-level folders, create some groups and users and poof, there goes another hour of your time.

Re: Ollama Turbo

#223

Post by leopoldj »

For production use of open weight models I'd use something like Amazon Bedrock, Google Vertex AI (which uses vLLM), or on-prem vLLM/SGLang. But for a quick assessment of a model as a developer, Ollama Turbo looks appealing. I find Google GCP incredibly user hostile and a nightmare to navigate quotas and stuff.

Re: Ollama Turbo

#224

Post by mark_l_watson »

mchiang wrote: post #196
buyucu wrote: post #192

Earlier quoted context omitted.

This kind of gaslighting is exactly why I stopped using Ollama. GGML library is llama.cpp. They are one and the same. Ollama made sense when llama.cpp was hard to use. Ollama does not have value preposition anymore.

It’s a different repo. https://github.com/ggml-org/ggml The models are implemented by Ollama https://github.com/ollama/ollama/tree/main/model/models I can say as a fact, for the gpt-oss model, we also implemented our own MXFP4 kernel. Benchmarked against the reference implementations to make sure Ollama is on par. We implemented harmony and tested it. This should significantly impact tool calling capability. Im not s…

Hello, thanks for answering questions here.

Is there a schedule for adding additional models to Turbo mode plan, in addition to gpt-oss 20/120b? I wanted to try your $20/month Turbo plan, but I would like to be able to experiment with a few other large models.

Re: Ollama Turbo

#225

Post by leopoldj »

mchiang wrote: post #35
jnmandal wrote: post #29

I see a lot of hate for ollama doing this kind of thing but also they remain one of the easiest to use solutions for developing and testing against a model locally. Sure, llama.cpp is the real thing, ollama is a wrapper... I would never want to use something like ollama in a production setting. But if I want to quickly get someone less technical up to speed to develop an LLM-enabled system and run qwen or w/e locally…

Thanks for the kind words. Since the new multimodal engine, Ollama has moved off of llama.cpp as a wrapper. We do continue to use the GGML library, and ask hardware partners to help optimize it. Ollama might look like a toy and what looks trivial to build. I can say, to keep its simplicity, we go through a deep amount of struggles to make it work with the experience we want. Simplicity is often overlooked, but we wan…

> Ollama has moved off of llama.cpp as a wrapper. We do continue to use the GGML library

Where can I learn more about this? llama.cpp is an inference application built using the ggml library. Does this mean, Ollama now has it's own code for what llama.cpp does?

Re: Ollama Turbo

#226

Post by jnmandal »

dcreater wrote: post #175
jnmandal wrote: post #162

Earlier quoted context omitted.

Honestly, I think it just depends. A few hours ago I wrote I would never want it for a production setting but actually if I was standing something up myself and I could just download headless ollama and know it would work. Hey, that would also be fine most likely. Maybe later on I'd revisit it from a devops perspective, and refactor deployment methodology/stack, etc. Maybe I'd benchmark it and realize its fine actual…

But its effectively equally easy to do the same with llama.cpp, vllm or modular.. (any differences are small enough that they either shouldn't cause the human much work or can very easily be delegated to AI)

Sure but if my some of the development team is using ollama locally b/c it was super easy to install, maybe I don't want to worry about maintaining a separate build chain for my prod env. Many startups are just wrapping or enabling LLMs and just need a running server. Who are we to say what is right use of their time and effort?

Re: Ollama Turbo

#228

Post by beowulfey »

supermatt wrote: post #91
coderatlarge wrote: post #86

Earlier quoted context omitted.

true but ignores handing over all your prompt traffic without any real legal protections as sama has pointed out: [1] https://californiarecorder.com/sam-altman-requires-ai-privil...

> OpenAI confirmed it has been preserving deleted and non permanent person chat logs since mid-Might 2025 in response to a federal court docket order > The order, embedded under and issued on Might 13, 2025, by U.S. Justice of the Peace Decide Ona T. Wang Is this some meme where “may” is being replaced with “might”, or some word substitution gone awry? I don’t get it.

auto correct gone awry

Re: Ollama Turbo

#229

Post by mark_l_watson »

smlacy wrote: post #15

Watching ollama pivot from a somewhat scrappy yet amazingly important and well designed open source project to a regular "for-profit company" is going to be sad. Thankfully, this may just leave more room for other open source local inference engines.

I don't blame them. As soon as they offer a few more models available with the Turbo mode I plan on subscribing to their Turbo plan for a couple of months - a buying them a coffee, or keeping the lights on kind of thing.

The Ollama app using the signed-in-only web search tool is really pretty good.

Re: Ollama Turbo

#230

Post by kristjansson »

mchiang wrote: post #154
kristjansson wrote: post #141

Earlier quoted context omitted.

> Ollama does not use llama.cpp anymore; > We do use GGML Sorry, but this is kind of hiding the ball. You don't use llama.cpp, you just ... use their core library that implements all the difficult bits, and carry a patchset on top of it? Why do you have to start with the first statement at all? "we use the core library from llama.cpp/ggml and implement what we think is a better interface and UX. we hope you like it a…

thanks, I'll take that feedback, but I do want to clarify that it's not from llama.cpp/ggml. It's from ggml-org/ggml. I supposed it's all interchangeable though, so thank you for it.

  % diff -ru ggml/src llama.cpp/ggml/src | grep -E '^(\+|\-) .*' | wc -l
      1445
i.e. as of time of writing +/- 1445 lines between the two, on about 175k total lines. a lot of which is the recent MXFP4 stuff.

Ollama is great software. It's integral to the broader diffusion of LLMs. You guys should be incredibly proud of it and the impact its had. I understand the current environment rewards bold claims, but the sense I get from some of your communications is "what's the boldest, strongest claim we can make that's still mostly technically true". As a potential user, taking those claims as true until closer evaluation reveals the discrepancy feels pretty bad, and keeps me firmly in the 'potential' camp.

Have the confidence in your software and the respect for your users to advertise your system as it is.

Post reply on HN