Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to setup a local coding agent on macOS

ikyle.me

Post reply on HN
Print view View original

21–30 of 150 posts

Re: How to setup a local coding agent on macOS

#21

Post by fridder »

vladgur wrote: post #19

I have used omlx.ai with great success to both download multiple mlx models (including gemma and qwen) suited for my hardware AND to be able to automagically launch both open-source and close-source (claude code, codex) harnesses using these models. All from a web or desktop UI You would not need to follow a blog post with omlx IMHO

It truly is the SOTA for local inference on mac. Even when there are regressions the dev(s) are insanely responsive. It is the most impressive opensource project I've seen in a awhile

Re: How to setup a local coding agent on macOS

#22

Post by iluvcommunism »

hkchad wrote: post #17
reddit_clone wrote: post #13

>64 GB Thats the rub. I have an M4 with 48G. I wonder if it is worth testing this out. My past attempts (with Ollama and various LLMs) were too slow to use.

I have a M5 MAX with 128, local models are toys compared to hosted ones. I've spent a lot of time and money trying to make it work even 1/2 as well.

[dead]

Re: How to setup a local coding agent on macOS

#24

Post by hanifbbz »

Here's a visual post for using LM Studio and VS Code (and Pi): https://blog.alexewerlof.com/p/local-llms-for-agentic-coding

One way or another local AI is the future. I actually find weaker models more interesting because it keeps me sharp (at the cost of velocity of course).

Re: How to setup a local coding agent on macOS

#29

Post by freehorse »

mft_ wrote: post #12
dofm wrote: post #6

Useful stuff in here that I wish I'd seen a few days ago :-) I am not convinced that the MTP setup for the QAT model adds very much in terms of speed on my M1 Max, but it is definitely worth experimenting with. Fiddling about with local models has done so much for my conceptual understanding of what is going on. FWIW and YMMV but I also found the Gemma 4 MTP head was occasionally breaking markup in Opencode, causing…

I found a marginal downside to Qwen3.6-35B-A3B-MTP vs. the non-MTP equivalent on an M1 Max. I’ll maybe experiment with settings further though.

And the upsides of using draft models for MOE models with so low number of active parameters (as here or as in the article) are quite low, compared to dense models where you can get enormous speedups. I would prefer running the dense 27b models with speculative decoding instead.

Re: How to setup a local coding agent on macOS

#30

Post by malkosta »

sleepybrett wrote: post #27

or you can just load up ollama, have it load a local model and point claude or opencode at it... is this article old? It's not. I'm not sure why he went through all the bother of llama.cpp

That was exactly my same question. Then I finished reading the post. The reason is pretty clear, and written in the post: it is faster than ollama+mlx.
Post reply on HN