Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

gist.github.com

Post reply on HN
Print view View original

121–130 of 132 posts

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#121

Post by DiabloD3 »

simondotau wrote: post #35
DiabloD3 wrote: post #29

Earlier quoted context omitted.

Ollama is quasi-open source. In some places in the source code they claim sole ownership of the code, when it is highly derivative of that in llama.cpp (having started its life as a llama.cpp frontend). They keep it the same license, however, MIT. There is no reason to use Ollama as an alternative to llama.cpp, just use the real thing instead.

If it’s MIT code derived from MIT code, in what way is its openness ”quasi”? Issues of attribution and crediting diminish the karma of the derived project, but I don’t see how it diminishes the level of openness.

FOSS licensing can only exist in terms of Copyright. Without Copyright, you cannot license FOSS. If something has an incorrect Copyright attribution, then the license can be viewed as invalid until this deficiency has been corrected (obv. depending on local laws, etc).

On top of this, it would not be unreasonable for the numerous authors of llama.cpp to issue DMCA takedown requests if Ollama is unwilling to correct it.

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#122

Post by ffsm8 »

DiabloD3 wrote: post #120
ffsm8 wrote: post #30

Earlier quoted context omitted.

> I don't think it does, but llama.cpp does I just checked their docs and can't see anything like it. Did you mistake the command to just download and load the model?

As a sibling comment answered you, it is `-hf`. And yes, it downloads the model, caches it, and then serves future loads of that model out of the cache if the file hasn't changed in the hf repo.

So I'm summary: no, it does not have an equivalent command either.

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#123

Post by solaire_oa »

milchek wrote: post #31

I tested briefly with a MacBook Pro m4 with 36gb. Run in LM Studio with open code as the frontend and it failed over and over on tool calls. Switched back to qwen. Anyone else on similar setup have better luck?

I can confirm that tool calls failed for me (Ubuntu server with charmbracelet/crush, if that matters)

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#124

Post by renewiltord »

zozbot234 wrote: post #58
renewiltord wrote: post #55

Just told Claude to sort it out and it ran it. 26 tok/s on the Mac mini I use for personal claw type program. Unusable for local agent but it’s okay.

Isn't 26 tok/s quite usable for a claw-like agent though? You can chat with it on a IM platform and get notified as soon as it replies, you're not dependent on real-time quick interaction.

For me it's too slow. Prefer using cloud agent. Can do more tasks.

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#125

Post by alfiedotwtf »

embedding-shape wrote: post #95
alfiedotwtf wrote: post #89

Earlier quoted context omitted.

After spending the past few weeks playing with different backends and models, I just can’t believe how buggy most models are. It seems to me that most model providers are not running/testing via the most used backends i.e Llama, Ollama etc because if they were, they would see how broken their release is. Tool calling is like the Achilles Heel where most will fail unless you either modify the system prompts or run via…

Just since I'm curious, what exact models and quantization are you using? In my own experience, anything smaller than ~32B is basically useless, and any quantization below Q8 absolutely trashes the models. Sure, for single use-cases, you could make use of a ~20B model if you fine-tune and have very narrow use-case, but at that point usually there are better solutions than LLMs in the first place. For something genera…

I haven’t tried any Qwen yet, but so far I’m sticking with gpt-oss-20B.

In terms of what I’m using, I’ve looked at anything that will fit on a MacBook Pro with 32Gb RAM (so with shared memory) - LFM2, Llama, Minstral, Ministral, Devstral, Phi, and Nemotron.

As for quantisation, I aim for the biggest that will fit while also not being too slow - so it all depends on the model. But I’ll skip a model if I can’t at least use a Q4_K_M.

Also, given that I also bump my context to at least 32K, because tooling sucks when the tooling definitions itself come close to 4096!

I can’t wait for RAM prices to come down!

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#127

Post by bicepjai »

milchek wrote: post #31

I tested briefly with a MacBook Pro m4 with 36gb. Run in LM Studio with open code as the frontend and it failed over and over on tool calls. Switched back to qwen. Anyone else on similar setup have better luck?

Yes same experience. Goes into loop mode where is sends same command again and again, till we kill it. This was Q_8 version on lmstudio

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#129

Post by jrm4 »

MarsIronPI wrote: post #118
the_lucifer wrote: post #99

Earlier quoted context omitted.

> Ollama user with the opposite question -- why not? What am I missing out on? I'm using it as the backend for playing with other frontend stuff and it seems to work just fine. Used to be an Ollama user. Everything that you cite as benefits for Ollama is what I was drawn to in the first place as well, then moved on to using llama.cpp directly. Apart from being extremely unethical, The issue is that they try to abstra…

> 1. It mangles GGUF files so other apps can't use them, and you can't access them either without a bunch of work on your end (had to script a way to unmangle these long sha-hashed file names) This is what pushed me away from Ollama. All I wanted was to scp a model from one machine to another so I didn't have to re-download it and waste bandwidth. But Ollama makes it annoying, so I switched to llama.cpp. I did also f…

Yup, pretty sure there are no licenses that say "you must upstream," just "if you upstream, do it openly."

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#130

Post by flux3125 »

dTal wrote: post #112
flux3125 wrote: post #52

Earlier quoted context omitted.

to be fair, llama.cpp has gotten much easier to use lately with llama-server -hf . That said, the need to compile it yourself is still a pretty big barrier for most people.

You don't need to compile it yourself though? Unless you want CUDA support on Linux I guess, dunno why you'd need such a silly thing though: https://github.com/ggml-org/llama.cpp/releases

> dunno why you'd need such a silly thing though

I'm not sure I follow, what alternative to CUDA on Linux offers similar performance?

Post reply on HN