Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ollama Turbo

ollama.com

Post reply on HN
Print view View original

191–200 of 251 posts

Re: Ollama Turbo

#191

Post by mchiang »

dcreater wrote: post #97
mchiang wrote: post #35

Earlier quoted context omitted.

Thanks for the kind words. Since the new multimodal engine, Ollama has moved off of llama.cpp as a wrapper. We do continue to use the GGML library, and ask hardware partners to help optimize it. Ollama might look like a toy and what looks trivial to build. I can say, to keep its simplicity, we go through a deep amount of struggles to make it work with the experience we want. Simplicity is often overlooked, but we wan…

But Ollama is a toy, it's meaningful for hobbyists and individuals to use locally like myself. Why would it be the right choice for anything more? AWS, vLLM, SGLang etc would be the solutions for enterprise I knew a startup that deployed ollama on a customers premises and when I asked them why, they had absolutely no good reason. Likely they did it because it was easy. That's not the "easy to use" case you want to so…

I can say trying many inference tools after the launch, many do not have the models implemented well, and especially OpenAI’s harmony.

Why does this matter? For this specific release, we benchmarked against OpenAI’s reference implementation to make sure Ollama is on par. We also spent a significant amount of time getting harmony implemented the way intended.

I know vLLM also worked hard to implement against the reference and have shared their benchmarks publicly.

Re: Ollama Turbo

#192

Post by buyucu »

mchiang wrote: post #35
jnmandal wrote: post #29

I see a lot of hate for ollama doing this kind of thing but also they remain one of the easiest to use solutions for developing and testing against a model locally. Sure, llama.cpp is the real thing, ollama is a wrapper... I would never want to use something like ollama in a production setting. But if I want to quickly get someone less technical up to speed to develop an LLM-enabled system and run qwen or w/e locally…

Thanks for the kind words. Since the new multimodal engine, Ollama has moved off of llama.cpp as a wrapper. We do continue to use the GGML library, and ask hardware partners to help optimize it. Ollama might look like a toy and what looks trivial to build. I can say, to keep its simplicity, we go through a deep amount of struggles to make it work with the experience we want. Simplicity is often overlooked, but we wan…

This kind of gaslighting is exactly why I stopped using Ollama.

GGML library is llama.cpp. They are one and the same.

Ollama made sense when llama.cpp was hard to use. Ollama does not have value preposition anymore.

Re: Ollama Turbo

#194

Post by jp1016 »

at this point, can i purchase the subscription directly from the model provider or hugging face and use it? or is this ollama attempt to become a provider like them.

Re: Ollama Turbo

#195

Post by miki123211 »

jnmandal wrote: post #29

I see a lot of hate for ollama doing this kind of thing but also they remain one of the easiest to use solutions for developing and testing against a model locally. Sure, llama.cpp is the real thing, ollama is a wrapper... I would never want to use something like ollama in a production setting. But if I want to quickly get someone less technical up to speed to develop an LLM-enabled system and run qwen or w/e locally…

> I would never want to use something like ollama in a production setting

If you can't get access to "real" datacenter GPUs for any reason and essentially do desktop, clientside deploys, it's your best bet.

It's not a common scenario, but a desktop with a 4090 or two is all you can get in some organizations.

Re: Ollama Turbo

#196

Post by mchiang »

buyucu wrote: post #192
mchiang wrote: post #35

Earlier quoted context omitted.

Thanks for the kind words. Since the new multimodal engine, Ollama has moved off of llama.cpp as a wrapper. We do continue to use the GGML library, and ask hardware partners to help optimize it. Ollama might look like a toy and what looks trivial to build. I can say, to keep its simplicity, we go through a deep amount of struggles to make it work with the experience we want. Simplicity is often overlooked, but we wan…

This kind of gaslighting is exactly why I stopped using Ollama. GGML library is llama.cpp. They are one and the same. Ollama made sense when llama.cpp was hard to use. Ollama does not have value preposition anymore.

It’s a different repo. https://github.com/ggml-org/ggml

The models are implemented by Ollama https://github.com/ollama/ollama/tree/main/model/models

I can say as a fact, for the gpt-oss model, we also implemented our own MXFP4 kernel. Benchmarked against the reference implementations to make sure Ollama is on par. We implemented harmony and tested it. This should significantly impact tool calling capability.

Im not sure if im feeding here. We really love what we do, and I hope it shows in our product, in Ollama’s design and in our voice to our community.

You don’t have to like Ollama. That’s subjective to your taste. As a maintainer, I certainly hope to have you as a user one day. If we don’t meet your needs and you want to use an alternative project, that’s totally cool too. It’s the power of having a choice.

Re: Ollama Turbo

#197

Post by jasonjmcghee »

apitman wrote: post #92
steren wrote: post #62

Earlier quoted context omitted.

> I would never want to use something like ollama in a production setting. We benchmarked vLLM and Ollama on both startup time and tokens per seconds. Ollama comes at the top. We hope to be able to publish these results soon.

Did you test multi-user cases?

Assuming this is equivalent to parallel sessions, I would hope so, this is like the entire point of vLLM

Re: Ollama Turbo

#198

Post by Hasnep »

dcreater wrote: post #185
Aurornis wrote: post #148

Earlier quoted context omitted.

> Its imperative we move away ASAP Why? If the tool works then use it. They’re not forcing you to use the cloud.

There are many, many FOSS apps that use Ollama as a dependency. If Ollama rugs, then all those projects suffer. Its a tale we seen played out many times. Redis is the most recent example.

Most apps that integrate with ollama that I've seen just have an OpenAI compatible API parameter which defaults to port 11434 which ollama uses, but can be changed easily. Is there a way to integrate ollama more deeply?

Re: Ollama Turbo

#200

Post by DiabloD3 »

decide1000 wrote: post #19

It was fun because it was open. Now it's just another brand seeking dollars.

Although it is open, its really just all code borrowed from llama.cpp.

If you want to see where the actual developers do the actual hard work, go use llama.cpp instead.

Post reply on HN