Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to setup a local coding agent on macOS

ikyle.me

Post reply on HN
Print view View original

51–60 of 150 posts

Re: How to setup a local coding agent on macOS

#51

Post by this_user »

echelon wrote: post #40
kingofthehill98 wrote: post #36

Earlier quoted context omitted.

Yeah, if the future is "Claude, think for me" I'm happy to stay at the good old present.

https://en.wikipedia.org/wiki/Is_Google_Making_Us_Stupid%3F https://newsletter.pessimistsarchive.org/p/when-educators-mo... New decade, same old argument. It's not > "Claude, think for me" It's > "Claude, be my subordinate and get this done for me" Instead of complaining on the sidelines, I'm getting a shit ton of work done.

> Instead of complaining on the sidelines, I'm getting a shit ton of work done.

Nah, you are just producing a bunch of slop and hope that nobody notices.

Re: How to setup a local coding agent on macOS

#52

Post by dofm »

freehorse wrote: post #29
mft_ wrote: post #12

Earlier quoted context omitted.

I found a marginal downside to Qwen3.6-35B-A3B-MTP vs. the non-MTP equivalent on an M1 Max. I’ll maybe experiment with settings further though.

And the upsides of using draft models for MOE models with so low number of active parameters (as here or as in the article) are quite low, compared to dense models where you can get enormous speedups. I would prefer running the dense 27b models with speculative decoding instead.

That is what I have learned, yes. Not tested the dense Qwen yet. IIRC the 31B Gemma was slow enough that I doubt MTP will help me much.

Re: How to setup a local coding agent on macOS

#53

Post by reenorap »

My biggest pet peeve with all these articles on local AI is the only thing they talk about is tokens per second. No one mentions the quality of the answers. No one. I don't mind waiting a little longer if the quality is better. Quickly serving me slop doesn't make it more useful. Are people really only looking at tokens per second?

Re: How to setup a local coding agent on macOS

#55

Post by dofm »

vladgur wrote: post #19

I have used omlx.ai with great success to both download multiple mlx models (including gemma and qwen) suited for my hardware AND to be able to automagically launch both open-source and close-source (claude code, codex) harnesses using these models. All from a web or desktop UI You would not need to follow a blog post with omlx IMHO

FWIW I have not, on a 64GB M1 Max, seen any advantage from oMLX specifically or MLX generally over GGUF with llama.cpp.

The Gemma 4 MLX builds I have found so far have been slower at the same quantisation and much slower with MTP.

The built-in web UI for llama.cpp is really quite good once you have chosen your model. Otherwise I quite like LM Studio for tinkering.

One thing I would say is that both Gemma-4 and Qwen 3.6 simply do not need a large chunk of the typical opencode system prompt. Better off without it.

Re: How to setup a local coding agent on macOS

#56

Post by dofm »

Dotnaught wrote: post #47
vladgur wrote: post #19

I have used omlx.ai with great success to both download multiple mlx models (including gemma and qwen) suited for my hardware AND to be able to automagically launch both open-source and close-source (claude code, codex) harnesses using these models. All from a web or desktop UI You would not need to follow a blog post with omlx IMHO

In case anyone is looking for a sandbox to go with oMLX and Pi: https://github.com/Dotnaught/pi-sandbox

This is useful. I'm still tinkering with Multipass VMs because I need the whole VM environment anyway and I'm on Sequoia. But I'd be interested if you did anything like that with Apple's container CLI instead; sooner or later I will have to upgrade to Tahoe because I want to play with the container CLI (and apfel).

Re: How to setup a local coding agent on macOS

#57

Post by dofm »

reddit_clone wrote: post #13

>64 GB Thats the rub. I have an M4 with 48G. I wonder if it is worth testing this out. My past attempts (with Ollama and various LLMs) were too slow to use.

Some of these models will be a bit of a squeeze at Q4_0 I suspect; almost certainly they will be using CPU. Probably the 31B Gemma will be too much. Maybe not the Gemma-4 26B QAT.

But if you just want to play around rather than code, you really might find the Gemma 4 12B model worth mucking about with just so you've gone through the steps. Especially if you want to muck about with image analysis or audio transcription.

If you're writing PHP I think you could even find it good enough. I've been modestly surprised. You can do that basic fiddling with the Edge AI Gallery app, which can enable thinking and has a customisable system prompt and some agent support.

You could also try the 14B Deepseek R1.

Honestly even if it is not good enough, if you are anything like me, I think you'll find that going through this process is really quite educational — it has made a lot of things more concrete for me in a way that I have found reassuring and valuable.

Re: How to setup a local coding agent on macOS

#58

Post by akman »

reenorap wrote: post #53

My biggest pet peeve with all these articles on local AI is the only thing they talk about is tokens per second. No one mentions the quality of the answers. No one. I don't mind waiting a little longer if the quality is better. Quickly serving me slop doesn't make it more useful. Are people really only looking at tokens per second?

That's fair. There are even many dimensions to define 'quality' which include use case (coding? writing? multimedia?) and prompt. I suppose if you ask testers to provide benchmarks with their analysis, that might hamper their desire to share.

Re: How to setup a local coding agent on macOS

#59

Post by mark_l_watson »

Nice writeup, thanks.

I run something very similar except for directly using pi as the agentic harness I use little-coder that wraps pi with reasonable defaults for running local models. Even though my local setup is a bit slow, it is a thrill to do real work completely locally.

Re: How to setup a local coding agent on macOS

#60

Post by reactordev »

Aurornis wrote: post #16

> The benchmark prompt was: > Write a compact Python function that parses a unified diff and returns the changed file paths. Then explain two edge cases. > Each benchmark generated about 128 tokens. Generating 128 tokens is probably not enough for good benchmark results. MTP speedup depends on how often the predicted tokens are accepted. In my experience, the very early output has a higher acceptance rate, so short t…

This is akin to saying “it runs on my machine” without actually examining the problem. Sad. You’re absolutely right that 128 tokens is nothing, it’s a little more than a hello response.
Post reply on HN