Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ollama Turbo

ollama.com

Post reply on HN
Print view View original

241–250 of 251 posts

Re: Ollama Turbo

#241

Post by buyucu »

zozbot234 wrote: post #237
buyucu wrote: post #236

Earlier quoted context omitted.

That pull request has been open for more than a year. The owner rebased multiple times but eventually gave up because Ollama devs just don't care.

That's not a helpful point of view. It's the contributors' job to keep a pull request up to date as the codebase evolves, a maintainer is under no obligation to accept a PR that has long become out of date and unmergeable.

The PR was in good shape. Ollama devs ignored it, and the original author rebased it multiple times. Since Ollama devs don't care, he just gave up after a while.

Ollama is in a very sad state. The project is dysfunctional.

Re: Ollama Turbo

#242

Post by buyucu »

mchiang wrote: post #196
buyucu wrote: post #192

Earlier quoted context omitted.

This kind of gaslighting is exactly why I stopped using Ollama. GGML library is llama.cpp. They are one and the same. Ollama made sense when llama.cpp was hard to use. Ollama does not have value preposition anymore.

It’s a different repo. https://github.com/ggml-org/ggml The models are implemented by Ollama https://github.com/ollama/ollama/tree/main/model/models I can say as a fact, for the gpt-oss model, we also implemented our own MXFP4 kernel. Benchmarked against the reference implementations to make sure Ollama is on par. We implemented harmony and tested it. This should significantly impact tool calling capability. Im not s…

This is exactly what I mean by gaslighting.

GGML is llama.cpp. It it developed by the same people as llama.cpp and powers everything llama.cpp does. You must know that. The fact that you are ignoring it very dishonest.

Re: Ollama Turbo

#243

Post by reissbaker »

irthomasthomas wrote: post #138
reissbaker wrote: post #127

Earlier quoted context omitted.

It was natively trained in FP4. Probably both to reduce VRAM usage at inference time (fits on a single H100), and to allow better utilization of B200s (which are especially fast for FP4).

Interesting, thanks. I didn't know you could even train at FP4 on H100s

It's impressive they got it to work — the lowest I'd heard of this far was native FP8 training.

Re: Ollama Turbo

#244

Post by davidron »

mattmaroon wrote: post #210
supermatt wrote: post #91

Earlier quoted context omitted.

> OpenAI confirmed it has been preserving deleted and non permanent person chat logs since mid-Might 2025 in response to a federal court docket order > The order, embedded under and issued on Might 13, 2025, by U.S. Justice of the Peace Decide Ona T. Wang Is this some meme where “may” is being replaced with “might”, or some word substitution gone awry? I don’t get it.

Or May in another language?

Or non native English speaker who pronounces "may" the same as "might" and didn't realize the difference?

It is maybe not coincidental that "may" and "might" mean nearly the same thing which bolsters the case for auto correct gone awry.

Re: Ollama Turbo

#245

Post by shepardrtc »

Aeolun wrote: post #133
shepardrtc wrote: post #34

Earlier quoted context omitted.

I think this offering is a perfectly reasonable option for them to make money. We all have bills to pay, and this isn't interfering with their open source project, so I don't see anything wrong with it.

> this isn't interfering with their open source project Wait until it makes significant amounts of money. Suddenly the priorities will be different. I don’t begrudge them wanting to make some money off it though.

You may be right, but I hope you aren't!

Re: Ollama Turbo

#246

Post by dcreater »

kristjansson wrote: post #230
mchiang wrote: post #154

Earlier quoted context omitted.

thanks, I'll take that feedback, but I do want to clarify that it's not from llama.cpp/ggml. It's from ggml-org/ggml. I supposed it's all interchangeable though, so thank you for it.

% diff -ru ggml/src llama.cpp/ggml/src | grep -E '^(\+|\-) .*' | wc -l 1445 i.e. as of time of writing +/- 1445 lines between the two, on about 175k total lines. a lot of which is the recent MXFP4 stuff. Ollama is great software. It's integral to the broader diffusion of LLMs. You guys should be incredibly proud of it and the impact its had. I understand the current environment rewards bold claims, but the sense I ge…

This is utterly damming.

Re: Ollama Turbo

#247

Post by benreesman »

kristjansson wrote: post #230
mchiang wrote: post #154

Earlier quoted context omitted.

thanks, I'll take that feedback, but I do want to clarify that it's not from llama.cpp/ggml. It's from ggml-org/ggml. I supposed it's all interchangeable though, so thank you for it.

% diff -ru ggml/src llama.cpp/ggml/src | grep -E '^(\+|\-) .*' | wc -l 1445 i.e. as of time of writing +/- 1445 lines between the two, on about 175k total lines. a lot of which is the recent MXFP4 stuff. Ollama is great software. It's integral to the broader diffusion of LLMs. You guys should be incredibly proud of it and the impact its had. I understand the current environment rewards bold claims, but the sense I ge…

I'm torn on this, I was a fan of the project from the very beginning and never sent any of my stuff upstream, so I'm less than a contributor but more than don't care, and it's still non-obvious how the split happened.

But the takeaway is pretty clearly that `llama.cpp`, `GGML`/`GGUF`, and generally `ggerganov`'s single-handedly Carmacking it when everyone thought it was impossible is all the value. I think a lot of people made Docker containers with `ggml`/`gguf` in them and one was like "we can make this a business if we realllllly push it".

Ollama as a hobby project or even a serious OSS project? With a cordial upstream relationship and massive attribution labels everywhere? Sure. Maybe even as a commercial thing that has a massive "Wouldn't Be Possible Without" page for it's OSS core upstream.

But like: startup company for making money that's (to all appearances) completely out of reach for the principles to ever do without totally `cp -r && git commit` repeatedly? It's complicated, a lot of stuff starts as a fork and goes off in a very different direction, and I got kinda nauseous and stopped paying attention at some point, but near as I can tell they're still just copying all the stuff they can't figure out how to do themselves on an ongoing basis without resolving the upstream drama?

It's like, in bounds barely I guess. I can't point to it being "this is strictly against the rules or norms", but it's bending everything to the absolute limit. It's not a zone I'd want to spend a lot of time in.

Re: Ollama Turbo

#248

Post by benreesman »

tarruda wrote: post #106
dcreater wrote: post #94

Called it. It's very unfortunate that the local inference community has aggregated around Ollama when it's clear that's not their long term priority or strategy. Its imperative we move away ASAP

Llama.cpp (library which ollama uses under the hoods) has its own server, and it is fully compatible with open-webui. I moved away from ollama in favor of llama-server a couple of months ago and never missed anything, since I'm still using the same UI.

I won't use `ollama` on principle. I use `llama-cli` and `llama-server` if I'm not linking `ggml`/`gguf` directly. It's like, two extra commands to use the one by the genius that wrote it and not the one that the guys just jacked it.

The models are on HuggingFace and downloading them is `uvx huggingface-cli`, the `GGUF` quants were `TheBloke` (with a grant from pmarca IIRC) for ages and now everyone does them (`unsloth` does a bunch of them).

Maybe I've got it twisted, but it seems to be that the people who actually do `ggml` aren't happy about it, and I've got their back on this.

Re: Ollama Turbo

#249

Post by benreesman »

moralestapia wrote: post #5

Ollama is great but I feel like Georgi Gerganov deserves way more credit for llama.cpp. He (almost) single-handedly brought LLMs to the masses. With the latest news of some AI engineers' compensation reaching up to a billion dollars, feels a bit unfair that Georgi is not getting a much larger slice of the pie.

`ggerganov` is one of the most under-rated and under-appreciated hackers maybe ever. His name belongs next to like Carmack and other people who made a new thing happen on PCs. And don't forget the shout out to `TheBloke` who like single-handedly bootstrapped the GGUF ecosystem of useful model quants (I think he had a grant from pmarca or something like that, so props to that too).

Re: Ollama Turbo

#250

Post by kristjansson »

benreesman wrote: post #247
kristjansson wrote: post #230

Earlier quoted context omitted.

% diff -ru ggml/src llama.cpp/ggml/src | grep -E '^(\+|\-) .*' | wc -l 1445 i.e. as of time of writing +/- 1445 lines between the two, on about 175k total lines. a lot of which is the recent MXFP4 stuff. Ollama is great software. It's integral to the broader diffusion of LLMs. You guys should be incredibly proud of it and the impact its had. I understand the current environment rewards bold claims, but the sense I ge…

I'm torn on this, I was a fan of the project from the very beginning and never sent any of my stuff upstream, so I'm less than a contributor but more than don't care, and it's still non-obvious how the split happened. But the takeaway is pretty clearly that `llama.cpp`, `GGML`/`GGUF`, and generally `ggerganov`'s single-handedly Carmacking it when everyone thought it was impossible is all the value. I think a lot of p…

To be clear I was comparing ggml-org/ggml to ggml-org/llama.cpp/ggml to respond to the earlier thing. Ollama carries an additional patchset on top of ggml-org/ggml.

> [ggml] is all the value

That’s what gets me about Ollama - they have real value too! Docker is just the kernel’s cgroups/chroots/iptables/… but it deserves a lot of credit for articulating and operating those on behalf of the user. Ollama deserves the same. But they’re consistently kinda weird about owning just that?

Post reply on HN