Ollama and gguf

Post by otabdeveloper4 » Thu, Aug 14, 2025, 5:32 AM UTC

A4ET8a8uTh0_v2 wrote: post #90

otabdeveloper4 wrote: post #71

Earlier quoted context omitted.

You shouldn't use Ollama as a beginner either. It comes with crazy begginer-hostile defaults out of the box.

Hmm? I would argue against that line of argumentation. It is ridiculously easy to start out of box and working. Once the user starts moving against obvious restrictions resulting from the trade-offs in defaults, they can move on to something more custom. Woulnd't that be the definition of beginner friendly? I am biased since I effectively started with Ollama as my main local llm so take this response for what it is.…

Post by pmarreck » Fri, Aug 15, 2025, 12:56 PM UTC

refulgentis wrote: post #35

hodgehog11 wrote: post #13

Earlier quoted context omitted.

I got to speak with some of the leads at Ollama and asked more or less this same question. The reason they abandoned llama.cpp is because it does not align with their goals. llama.cpp is designed to rapidly adopt research-level optimisations and features, but the downside is that reported speeds change all the time (sometimes faster, sometimes slower) and things break really often. You can't hope to establish contrac…

This is a good handwave-y answer for them but truth is they've always been allergic to ever mentioning llama.cpp, even when legally required, they made a political decision instead of an engineering one, and now justify it to themselves and you by handwaving about it somehow being less stable than the core of it, which they still depend on. A lot of things happened to get to the point they're getting called out aggre…

Post by pmarreck » Fri, Aug 15, 2025, 2:19 PM UTC

woodrowbarlow wrote: post #94

pmarreck wrote: post #93

Earlier quoted context omitted.

> but truth is they've always been allergic to ever mentioning llama.cpp well, it's literally named "o"-llama, so...

"llama" is the name of the open-weights model released by meta for which llama.cpp (not by meta) was originally designed. llama.cpp now supports many models, not just llama. they're separate projects with separate authors.

Post by jychang » Sun, Aug 24, 2025, 11:29 AM UTC

MarkSweep wrote: post #96

jychang wrote: post #44

Earlier quoted context omitted.

? llamacpp> ls -l \*llama\* -rwxr-xr-x 1 root root 2505480 Aug 7 05:06 libllama.so -rwxr-xr-x 1 root root 5092024 Aug 7 05:23 llama-server That's a terrible excuse, Llama.cpp is just 7.5 megabytes. You can easily ship a couple copies of that. The current ollama for windows download is 700MB. I don't buy it. They're not willing to make an 700MB download a few megabytes bigger to ~730MB, but they are willing to support…

Sorry, I forgot to include in my comment this part: If you include multiple version of llama, and each of those llama version depends on different GPU libraries, that could balloon the download size. If these GPU libraries change rarely, then yes, you are correct, it might not be a problem.