Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to setup a local coding agent on macOS

ikyle.me

Post reply on HN
Print view View original

111–120 of 150 posts

Re: How to setup a local coding agent on macOS

#111

Post by leemoore »

hkchad wrote: post #17
reddit_clone wrote: post #13

>64 GB Thats the rub. I have an M4 with 48G. I wonder if it is worth testing this out. My past attempts (with Ollama and various LLMs) were too slow to use.

I have a M5 MAX with 128, local models are toys compared to hosted ones. I've spent a lot of time and money trying to make it work even 1/2 as well.

I have the same processor and ram. The dense 30b ish Gemma/Qwen really don't break 10 TPS with or without MTP. MOE's in this range feel more usable if they are smart enough for your work. Probably would still use hosted versions of these over local unless. MOE's feel somewhere between sonnet 3.5 and 3.7 to me. Dense feels between sonnet 3.7 and 4 in basic coding or local agentic capabilities (not close to those in chat or world knowledge)

Re: How to setup a local coding agent on macOS

#112

Post by d4rkp4ttern »

It’s relatively simple to use llama.cpp/server to spin up a local LLM to work with Claude Code or Codex-CLI. The required llama server settings are often scattered all over so I maintain a set of instructions here for several popular open LLMs:

https://pchalasani.github.io/claude-code-tools/integrations/...

Re: How to setup a local coding agent on macOS

#114

Post by ricardobeat »

d4rkp4ttern wrote: post #112

It’s relatively simple to use llama.cpp/server to spin up a local LLM to work with Claude Code or Codex-CLI. The required llama server settings are often scattered all over so I maintain a set of instructions here for several popular open LLMs: https://pchalasani.github.io/claude-code-tools/integrations/...

Do you use that as a daily driver? Claude Code' prompt is huge and causes you to spend a long, long time on prompt processing for local models, then running out of context shortly after.

Re: How to setup a local coding agent on macOS

#115

Post by dominotw »

echelon wrote: post #40
kingofthehill98 wrote: post #36

Earlier quoted context omitted.

Yeah, if the future is "Claude, think for me" I'm happy to stay at the good old present.

https://en.wikipedia.org/wiki/Is_Google_Making_Us_Stupid%3F https://newsletter.pessimistsarchive.org/p/when-educators-mo... New decade, same old argument. It's not > "Claude, think for me" It's > "Claude, be my subordinate and get this done for me" Instead of complaining on the sidelines, I'm getting a shit ton of work done.

> I'm getting a shit ton of work done.

maybe you stopped thinking too much that you dont regonize that you are just producing slop that no one cares about.

AI is now getting humans to produce slop

Re: How to setup a local coding agent on macOS

#116

Post by alexwwang »

I wonder if these local model could really solve problems especially for users that aren’t experts on a given coding language. I am not sure that, more than inline auto completion and unit implementation, are these model capable of designing and composing tech specs that really work.

Re: How to setup a local coding agent on macOS

#117

Post by d4rkp4ttern »

ricardobeat wrote: post #114
d4rkp4ttern wrote: post #112

It’s relatively simple to use llama.cpp/server to spin up a local LLM to work with Claude Code or Codex-CLI. The required llama server settings are often scattered all over so I maintain a set of instructions here for several popular open LLMs: https://pchalasani.github.io/claude-code-tools/integrations/...

Do you use that as a daily driver? Claude Code' prompt is huge and causes you to spend a long, long time on prompt processing for local models, then running out of context shortly after.

Yes CC prompt can be ~30K tokens. I definitely do not use this as a daily driver. I did use it a few times for sensitive document work with Qwen3.6 MOE.

Re: How to setup a local coding agent on macOS

#119

Post by jmkni »

reenorap wrote: post #53

My biggest pet peeve with all these articles on local AI is the only thing they talk about is tokens per second. No one mentions the quality of the answers. No one. I don't mind waiting a little longer if the quality is better. Quickly serving me slop doesn't make it more useful. Are people really only looking at tokens per second?

The quality is obviously much worse, but still useful as a reference if you generally know what you are doing

It solve the "I'm coding on the plane and need to look up this thing I've forgotten" problem, for me at least

Re: How to setup a local coding agent on macOS

#120

Post by dofm »

amboo7 wrote: post #98
fouc wrote: post #96

Earlier quoted context omitted.

what? you're saying both MLX and MTP have been slower for your mac?

I also have an M1 Max 64GB: Qwen 3.6 benefits from MTP (after rounds of parameter optimization). MLX was unstable (haven't tried it recently), faster at TG but slower at PP, so inconclusive.

Yeah. I have not really tinkered much with parameter optimisation for the 35B model with MTP. Would be interested to see what you've found.

I'm using the GGUF too; it appears slightly faster in llama.cpp now than current LM Studio but it's not clear to me if that is down to LM Studio having a little more code overhead, older llama.cpp under the hood, or just parameter differences.

Post reply on HN