91–100 of 150 posts
I wonder why something like LM Studio didn't work for the author?
That’s what I was wondering, lm studio and draw things are easy to use apps that handle much of the cruft for you
Earlier quoted context omitted.
That's the direct link: https://ikyle.me/blog/2026/how-to-setup-a-local-coding-agent...
Note this is cut to just before the model responds, so not a great way for people to judge the real-time feel of this.
Plus a followup one where you see me type the question in and press enter (though that video is with Qwen 3.6, not Gemma 4) https://x.com/Freerunnering/status/2065354101878055038
Earlier quoted context omitted.
That was exactly my same question. Then I finished reading the post. The reason is pretty clear, and written in the post: it is faster than ollama+mlx.
how much faster?
Original video: https://x.com/Freerunnering/status/2065275403548168398
And in the blog post there is a table showing the different speeds I got from different engines.
Slowest combo was 38.1 tk/s, and the fastest was 72.2 tk/s. All from "the same" model.
I have used omlx.ai with great success to both download multiple mlx models (including gemma and qwen) suited for my hardware AND to be able to automagically launch both open-source and close-source (claude code, codex) harnesses using these models. All from a web or desktop UI You would not need to follow a blog post with omlx IMHO
FWIW I have not, on a 64GB M1 Max, seen any advantage from oMLX specifically or MLX generally over GGUF with llama.cpp. The Gemma 4 MLX builds I have found so far have been slower at the same quantisation and much slower with MTP. The built-in web UI for llama.cpp is really quite good once you have chosen your model. Otherwise I quite like LM Studio for tinkering. One thing I would say is that both Gemma-4 and Qwen 3…
>64 GB Thats the rub. I have an M4 with 48G. I wonder if it is worth testing this out. My past attempts (with Ollama and various LLMs) were too slow to use.
I have a M5 MAX with 128, local models are toys compared to hosted ones. I've spent a lot of time and money trying to make it work even 1/2 as well.
I've dabbled with Qwen 3.x and Gemma 4 models a bit. They are alright but not that impressive. And my mac gets super hot if I use them for extended periods of time. It's just not very nice to use locally.
Earlier quoted context omitted.
FWIW I have not, on a 64GB M1 Max, seen any advantage from oMLX specifically or MLX generally over GGUF with llama.cpp. The Gemma 4 MLX builds I have found so far have been slower at the same quantisation and much slower with MTP. The built-in web UI for llama.cpp is really quite good once you have chosen your model. Otherwise I quite like LM Studio for tinkering. One thing I would say is that both Gemma-4 and Qwen 3…
what? you're saying both MLX and MTP have been slower for your mac?
oMLX ( https://github.com/jundot/omlx ) makes running the mlx inference server quite easy for those interested in UI-based hosting. oMLX also supports mtp or dflash drafting.
>64 GB Thats the rub. I have an M4 with 48G. I wonder if it is worth testing this out. My past attempts (with Ollama and various LLMs) were too slow to use.
omlx + gemma 12b 6 bit + pi
it’s feasible for sure
MoEs for speed (qwen 35b, cohere 30b, gemma 26b)
Dense for more methodical work (qwen 27b [reigning champ], gemma 31b, gemma 12b)
MoE i recommend 5bit+
Dense i think 4 bit is okay
Play with your context size, you don’t really need that much, have lazy loading for tools and mcps
my pi extensions for anyone looking for a skinny quick setup, i have use `--no-skills` right now too:
"npm:pi-codex-goal",
"npm:pi-simplify",
"npm:pi-mcp-adapter",
"git:github.com/elpapi42/pi-minimal-subagent",
"npm:@wierdbytes/pi-statusline",
"npm:@aliou/pi-guardrails",
"npm:pi-lens",
"npm:@juicesharp/rpiv-todo",
"npm:pi-hashline-readmap",
"npm:@mrclrchtr/supi-review",
"npm:pi-cmux",
"npm:@mrclrchtr/supi-context",
"npm:pi-tool-search"