Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

gist.github.com

Post reply on HN
Print view View original

61–70 of 132 posts

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#61

Post by colechristensen »

Aurornis wrote: post #56

If this is your first time using open weight models right after release, know that there are always bugs in the early implementations and even quantizations. Every project races to have support on launch day so they don’t lose users, but the output you get may not be correct. There are already several problems being discovered in tokenizer implementations and quantizations may have problems too if they use imatrix. S…

You seem like you know what you're talking about... what inference engine should I use? (linux, 4090)

I keep having "I tried it but it sucks" issues mostly around tool calling and it's not clear if it's the model or ollama. And not one model in particular, any of them really.

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#62

Post by armchairhacker »

Aurornis wrote: post #49
mark_l_watson wrote: post #42

The article has a few good tips for using Ollama. Perhaps it should note that the Gemma 4 models are not really trained for strong performance with coding agents like OpenCode, Claude Code, pi, etc. The Gemma 4 models are excellent for applications requiring tool use, data extraction to JSON, etc. I asked Gemini Pro about this earlier and Gemini Pro recommended qwen 3.5 models specifically for coding, and backed that…

> I asked Gemini Pro about this earlier and Gemini Pro recommended qwen 3.5 models specifically for coding, and backed that up with interesting material on training. The Gemma models were literally released yesterday. You can’t ask LLMs for advice on these topics and get accurate information. Please don’t repeat LLM-sourced answers as canonical information

LLMs can search the web. Although I don’t trust the LLM (or someone repeating its claim) without quotes and URLs to where it got the information.

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#63

Post by vardalab »

colechristensen wrote: post #61
Aurornis wrote: post #56

If this is your first time using open weight models right after release, know that there are always bugs in the early implementations and even quantizations. Every project races to have support on launch day so they don’t lose users, but the output you get may not be correct. There are already several problems being discovered in tokenizer implementations and quantizations may have problems too if they use imatrix. S…

You seem like you know what you're talking about... what inference engine should I use? (linux, 4090) I keep having "I tried it but it sucks" issues mostly around tool calling and it's not clear if it's the model or ollama. And not one model in particular, any of them really.

just use openrouter or google ai playground for the first week till bugs are ironed out. You still learn the nuances of the model and then yuu can switch to local. In addition you might pickup enough nuance to see if quantization is having any effect

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#64

Post by jrm4 »

easygenes wrote: post #5

Why is ollama so many people’s go-to? Genuinely curious, I’ve tried it but it feels overly stripped down / dumbed down vs nearly everything else I’ve used. Lately I’ve been playing with Unsloth Studio and think that’s probably a much better “give it to a beginner” default.

Ollama user with the opposite question -- why not? What am I missing out on? I'm using it as the backend for playing with other frontend stuff and it seems to work just fine.

And as someone running at 16gb card, I'm especially curious as to if I'm missing out on better performance?

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#65

Post by Aurornis »

colechristensen wrote: post #61
Aurornis wrote: post #56

If this is your first time using open weight models right after release, know that there are always bugs in the early implementations and even quantizations. Every project races to have support on launch day so they don’t lose users, but the output you get may not be correct. There are already several problems being discovered in tokenizer implementations and quantizations may have problems too if they use imatrix. S…

You seem like you know what you're talking about... what inference engine should I use? (linux, 4090) I keep having "I tried it but it sucks" issues mostly around tool calling and it's not clear if it's the model or ollama. And not one model in particular, any of them really.

I don’t know if any of engines are fully tested yet.

For new LLMs I get in the habit of building llama.cpp from upstream head and checking for updated quantizations right before I start using it. You can also download llama.cpp CI builds from their release page but on Linux it’s easy to set up a local build.

If you don’t want to be a guinea pig for untested work then the safe option would be to wait 2-3 weeks

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#66

Post by jrm4 »

redrove wrote: post #3

There is virtually no reason to use Ollama over LM Studio or the myriad of other alternatives. Ollama is slower and they started out as a shameless llama.cpp ripoff without giving credit and now they “ported” it to Go which means they’re just vibe code translating llama.cpp, bugs included.

Do y'all mean backend or the Ollama frontend or both? I find it trivially easy to sub in my local Ollama api thing in virtually all of the interesting frontend things. I'm quite curious about the "why not Ollama" here.

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#67

Post by jasonjmcghee »

abroadwin wrote: post #46
jasonjmcghee wrote: post #41

Earlier quoted context omitted.

Haven't had time to try yet, but heard from others that they needed to update both the main and runtime versions for things to work.

Even with the latest version of LM Studio and the latest runtimes I find that tool use fails 100% of the time with the following error: Error rendering prompt with jinja template: "Cannot apply filter "upper" to type: UndefinedValue". EDIT: The issue is addressed in LM Studio 0.4.9 (build 1), which auto-update wasn't picking up for me for some reason.

I googled it- supposed fixed template

https://github.com/ggml-org/llama.cpp/issues/21347#issuecomm...

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#68

Post by Yukonv »

anonyfox wrote: post #44

M5 air here with 32gb ram and 10/10 cores. Anyone got some luck with mlx builds on oMLX so far? Not at my machine right now and would love to know if these models already work including tool calling

The latest release v0.3.2 has partial support, generation is supported but not all special tokens are handled. I've done some personal testing to add tool calling and thinking support. https://github.com/Yukon/omlx

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#69

Post by SparkyMcUnicorn »

mark_l_watson wrote: post #60
Aurornis wrote: post #49

Earlier quoted context omitted.

> I asked Gemini Pro about this earlier and Gemini Pro recommended qwen 3.5 models specifically for coding, and backed that up with interesting material on training. The Gemma models were literally released yesterday. You can’t ask LLMs for advice on these topics and get accurate information. Please don’t repeat LLM-sourced answers as canonical information

I spent two hours doing my own research before asking for Gemini’s analysis, which reinforced my own opinion that the gemini models historically have not been trained and target for agentic coding use. Have you tried using the new Gemma 4 models with agentic coding tools?If you do, you might end up agreeing with me.

I've found my research on certain topics like this becoming less reliable these days, compared to just trying it out to form an opinion.

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#70

Post by mark_l_watson »

SparkyMcUnicorn wrote: post #69
mark_l_watson wrote: post #60

Earlier quoted context omitted.

I spent two hours doing my own research before asking for Gemini’s analysis, which reinforced my own opinion that the gemini models historically have not been trained and target for agentic coding use. Have you tried using the new Gemma 4 models with agentic coding tools?If you do, you might end up agreeing with me.

I've found my research on certain topics like this becoming less reliable these days, compared to just trying it out to form an opinion.

I wasn’t very clear, sorry. By my ‘own research’ I meant spending 90 minutes experimenting with Gemma 4 models for tool use (good results!) and a half hour using with pi and OpenCode (I didn’t get good results, yet.)
Post reply on HN