Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

gist.github.com

Post reply on HN
Print view View original

51–60 of 132 posts

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#51

Post by renewiltord »

mark_l_watson wrote: post #42

The article has a few good tips for using Ollama. Perhaps it should note that the Gemma 4 models are not really trained for strong performance with coding agents like OpenCode, Claude Code, pi, etc. The Gemma 4 models are excellent for applications requiring tool use, data extraction to JSON, etc. I asked Gemini Pro about this earlier and Gemini Pro recommended qwen 3.5 models specifically for coding, and backed that…

Oh yeah absolute genius. I asked GPT-2 about Claude Opus 4.6 and it said “this is not a recommendation. You might get some benefits from Opus… but this is not what you want”. Damn, real wisdom from the OG there. What a legend

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#52

Post by flux3125 »

diflartle wrote: post #17
easygenes wrote: post #5

Why is ollama so many people’s go-to? Genuinely curious, I’ve tried it but it feels overly stripped down / dumbed down vs nearly everything else I’ve used. Lately I’ve been playing with Unsloth Studio and think that’s probably a much better “give it to a beginner” default.

Ollama is good enough to dabble with, and getting a model is as easy as ollama pull vs figuring it out by yourself on hugging face and trying to make sense on all the goofy letters and numbers between the forty different names of models, and not needing a hugging face account to download. So you start there and eventually you want to get off the happy path, then you need to learn more about the server and it's all so…

to be fair, llama.cpp has gotten much easier to use lately with llama-server -hf . That said, the need to compile it yourself is still a pretty big barrier for most people.

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#53

Post by zozbot234 »

lambda wrote: post #23
krzyk wrote: post #15

Earlier quoted context omitted.

By desk you mean that "Mac mini"? Because it is pricey. In my country it is 1000 USD (from Apple for basic M4 with 24GB). My desk was 1/5th of that price. And considering that this Mac mini won't be doing anything else is there a reason why not just buy subscription from Claude, OpenAI, Google, etc.? Are those open models more performant compared to Sonnet 4.5/4.6? Or have at least bigger context?

Right now, open models that run on hardware that costs under $5000 can get up to around the performance of Sonnet 3.7. Maybe a bit better on certain tasks if you fine tune them for that specific task or distill some reasoning ability from Opus, but if you look at a broad range of benchmarks, that's about where they land in performance. You can get open models that are competitive with Sonnet 4.6 on benchmarks (though…

You don't need that much VRAM to run the very largest models, these are MoE models where only a small fraction is being computed with at any given time. If you plan to run with multiple GPUs and have enough PCIe lanes (such as with a proper HEDT platform) CPU-GPU transfers start to become a bit less painful. More importantly, streaming weights from disk becomes feasible, which lets you save on expensive RAM. The big labs only avoid this because it costs power at scale compared to keeping weights in DRAM, but that aside it's quite sound.

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#54

Post by zozbot234 »

Aurornis wrote: post #49
mark_l_watson wrote: post #42

The article has a few good tips for using Ollama. Perhaps it should note that the Gemma 4 models are not really trained for strong performance with coding agents like OpenCode, Claude Code, pi, etc. The Gemma 4 models are excellent for applications requiring tool use, data extraction to JSON, etc. I asked Gemini Pro about this earlier and Gemini Pro recommended qwen 3.5 models specifically for coding, and backed that…

> I asked Gemini Pro about this earlier and Gemini Pro recommended qwen 3.5 models specifically for coding, and backed that up with interesting material on training. The Gemma models were literally released yesterday. You can’t ask LLMs for advice on these topics and get accurate information. Please don’t repeat LLM-sourced answers as canonical information

It's not just LLM sourced though, folks have literally tried this after the release with the 26A4B model and it wasn't very good. Maybe the dense ~31B model is worthwhile though.

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#56

Post by Aurornis »

If this is your first time using open weight models right after release, know that there are always bugs in the early implementations and even quantizations.

Every project races to have support on launch day so they don’t lose users, but the output you get may not be correct. There are already several problems being discovered in tokenizer implementations and quantizations may have problems too if they use imatrix.

So you’re going to see a lot of “I tried it but it sucks because it can’t even do tool calls” and other reports about how the models don’t work at all in the coming weeks from people who don’t realize they were using broken implementations.

If you want to try cutting edge open models you need to be ready to constantly update your inference engine and check your quantization for updates and re-download when it’s changed. The mad rush to support it on launch day means everything gets shipped as soon as it looks like it can produce output tokens, not when it’s tested to be correct.

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#57

Post by Aurornis »

zozbot234 wrote: post #54
Aurornis wrote: post #49

Earlier quoted context omitted.

> I asked Gemini Pro about this earlier and Gemini Pro recommended qwen 3.5 models specifically for coding, and backed that up with interesting material on training. The Gemma models were literally released yesterday. You can’t ask LLMs for advice on these topics and get accurate information. Please don’t repeat LLM-sourced answers as canonical information

It's not just LLM sourced though, folks have literally tried this after the release with the 26A4B model and it wasn't very good. Maybe the dense ~31B model is worthwhile though.

Many Gemma implementations are or were broken on launch day. The first attempts to fix llama.cpp’s tokenizer were merged hours ago.

Everyone hated Qwen3.5 at launch too because so many implementations were broken and couldn’t do tool calling.

You need to ignore social media “I tried this and it sucks” echo chambers for new model releases.

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#58

Post by zozbot234 »

renewiltord wrote: post #55

Just told Claude to sort it out and it ran it. 26 tok/s on the Mac mini I use for personal claw type program. Unusable for local agent but it’s okay.

Isn't 26 tok/s quite usable for a claw-like agent though? You can chat with it on a IM platform and get notified as soon as it replies, you're not dependent on real-time quick interaction.

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#59

Post by Aurornis »

milchek wrote: post #31

I tested briefly with a MacBook Pro m4 with 36gb. Run in LM Studio with open code as the frontend and it failed over and over on tool calls. Switched back to qwen. Anyone else on similar setup have better luck?

Tool calls falling is a problem with the inference engine’s implementation and/or the quant. Update and try again in a few days.

This is how all open weight model launches go.

Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini

#60

Post by mark_l_watson »

Aurornis wrote: post #49
mark_l_watson wrote: post #42

The article has a few good tips for using Ollama. Perhaps it should note that the Gemma 4 models are not really trained for strong performance with coding agents like OpenCode, Claude Code, pi, etc. The Gemma 4 models are excellent for applications requiring tool use, data extraction to JSON, etc. I asked Gemini Pro about this earlier and Gemini Pro recommended qwen 3.5 models specifically for coding, and backed that…

> I asked Gemini Pro about this earlier and Gemini Pro recommended qwen 3.5 models specifically for coding, and backed that up with interesting material on training. The Gemma models were literally released yesterday. You can’t ask LLMs for advice on these topics and get accurate information. Please don’t repeat LLM-sourced answers as canonical information

I spent two hours doing my own research before asking for Gemini’s analysis, which reinforced my own opinion that the gemini models historically have not been trained and target for agentic coding use.

Have you tried using the new Gemma 4 models with agentic coding tools?If you do, you might end up agreeing with me.

Post reply on HN