Huge Claude user here… can someone help me set some realistic expectations if I bought a Mac mini and spun one up? I use Claude primarily for dev work and Home Lab projects. Are the open models good enough to run locally and replace the Claude workload? Or am I better off with my $20/mo Claude subscription?
April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini
81–90 of 132 posts
Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini
#82[dead]
Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini
#83[dead]
Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini
#84Huge Claude user here… can someone help me set some realistic expectations if I bought a Mac mini and spun one up? I use Claude primarily for dev work and Home Lab projects. Are the open models good enough to run locally and replace the Claude workload? Or am I better off with my $20/mo Claude subscription?
They are good for small tasks but you would not be able to use it like you use Claude and most likely be disappointed. But also, I do not know how you use claude.
There are many services online which offer hosted services for these models, my advice for anyone who is thinking about buying hardware to self host this is to try those first, that way you can get an impression of the capabilities and limitations of those models before you commit to buying hardware
Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini
#85If this is your first time using open weight models right after release, know that there are always bugs in the early implementations and even quantizations. Every project races to have support on launch day so they don’t lose users, but the output you get may not be correct. There are already several problems being discovered in tokenizer implementations and quantizations may have problems too if they use imatrix. S…
You seem like you know what you're talking about... what inference engine should I use? (linux, 4090) I keep having "I tried it but it sucks" issues mostly around tool calling and it's not clear if it's the model or ollama. And not one model in particular, any of them really.
I've had really good success with LMStudio and GLM 4.7 Flash and the Zed editor which has a baked in integration with LMStudio. I am able to one-shot whole projects this way, and it seems to be constantly improving. Some update recently even allowed the agent to ask me if it can do a "research" phase - so it'll actually reach out to website and read docs and code from github if you allow it. GLM 4.7 flash has been the most adept at tool calling I've found, but the Qwen 3 and 3.5 models are also fairly good, though run into more snags than I've seen with GLM 4.7 flash.
Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini
#86Why is ollama so many people’s go-to? Genuinely curious, I’ve tried it but it feels overly stripped down / dumbed down vs nearly everything else I’ve used. Lately I’ve been playing with Unsloth Studio and think that’s probably a much better “give it to a beginner” default.
What I really don't get is why more people don't talk about LMStudio, I switched to it months ago and it seems like a straight upgrade.
How does LMStudio compare to Unsloth Studio?
Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini
#87Weird that the steps are for "Gemma 4 12b", which does not exist, and then switches to 26b midway through.
There's also a step to verify that it doesn't fit on the GPU with ollama ps showing "14%/86% CPU/GPU". Doesn't this mean you'll have really bad performance?
Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini
#88Earlier quoted context omitted.
Right now, open models that run on hardware that costs under $5000 can get up to around the performance of Sonnet 3.7. Maybe a bit better on certain tasks if you fine tune them for that specific task or distill some reasoning ability from Opus, but if you look at a broad range of benchmarks, that's about where they land in performance. You can get open models that are competitive with Sonnet 4.6 on benchmarks (though…
You don't need that much VRAM to run the very largest models, these are MoE models where only a small fraction is being computed with at any given time. If you plan to run with multiple GPUs and have enough PCIe lanes (such as with a proper HEDT platform) CPU-GPU transfers start to become a bit less painful. More importantly, streaming weights from disk becomes feasible, which lets you save on expensive RAM. The big…
While you can run with weights in RAM or even disk, it gets a lot slower; even though on any given token a fraction of the weights are used, that can change with each token, so there is a lot of traffic to transfer weights to the GPU, which is a lot slower than if it's directly in GPU RAM. And even more slower if you stream from disk. Possible, yes, and maybe OK for some purposes, but you might find it painfully slow.
Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini
#89Earlier quoted context omitted.
You seem like you know what you're talking about... what inference engine should I use? (linux, 4090) I keep having "I tried it but it sucks" issues mostly around tool calling and it's not clear if it's the model or ollama. And not one model in particular, any of them really.
For the specific issue parent is talking about, you really need to give various tools a try yourself, and if you're getting really shit results, assume it's the implementation that is wrong, and either find an existing bug tracker issue or create a new one. Same thing happened when GPT-OSS launched, bunch of projects had "day-1" support, but in reality it just meant you could load the model basically, a bunch of them…
After spending the past few weeks playing with different backends and models, I just can’t believe how buggy most models are.
It seems to me that most model providers are not running/testing via the most used backends i.e Llama, Ollama etc because if they were, they would see how broken their release is.
Tool calling is like the Achilles Heel where most will fail unless you either modify the system prompts or run via proxies so you can inject/munge the request/reply.
Like seriously… how many billions and billions (actually we saw one >800 billion evaluation last week, so almost a whole trillion) goes into AI development and yet 99.999% of all models from the big names do not work straight out of the box with the most common backends. Blows my mind!
Re: April 2026 TLDR Setup for Ollama and Gemma 4 26B on a Mac mini
#90Earlier quoted context omitted.
Ollama is good enough to dabble with, and getting a model is as easy as ollama pull vs figuring it out by yourself on hugging face and trying to make sense on all the goofy letters and numbers between the forty different names of models, and not needing a hugging face account to download. So you start there and eventually you want to get off the happy path, then you need to learn more about the server and it's all so…
to be fair, llama.cpp has gotten much easier to use lately with llama-server -hf . That said, the need to compile it yourself is still a pretty big barrier for most people.
I started with ollama and now I'm using llama.cpp/llama-server's Router Mode that allows you to manage multiple models through a single server instance.
One thing I haven't figured out: Subjectively, it feels like ollama's model loading was nearly instant, while I feel like I'm always waiting for llama.cpp to load models, but that doesn't make sense because it's ultimately the same software. Maybe I should try ollama again to convince myself that I'm not crazy and that ollama's model loading wasn't actually instant.