I use LM Studio with the local server it ships and connect it to opencode. Takes 2 min to setup
[dead]
How to setup a local coding agent on macOS
101–110 of 150 posts
Re: How to setup a local coding agent on macOS
#102I wrote a similar post some time ago just used ollama and opencode https://blog.kulman.sk/running-local-llm-coding-server/
Ollama is not a good choice - https://sleepingrobots.com/dreams/stop-using-ollama/
As for oprncode, doesn't the system prompt eat too much of the context? Local models are really constraint in regards contex, and opencode AFAIR uses a 10k of it or some thing close.
Re: How to setup a local coding agent on macOS
#103>64 GB Thats the rub. I have an M4 with 48G. I wonder if it is worth testing this out. My past attempts (with Ollama and various LLMs) were too slow to use.
People are using 3090 (24GB) to run models, and it is the most cost effective way to run the. Yes, it is 2x faster, but memory wise you surely can spend 24gb on llm.
Also there are smaller, still usefull models that can run on 8GB or less.
Re: How to setup a local coding agent on macOS
#104FYI you can open Claude code in the terminal, point it at this article and just tell it to "do it", if you're feeling extra lazy
This is the way. I'm not Googling much of anything anymore. 9/10 times the information is awful, it's hard to parse out of whatever other spam it's surrounded by. Meanwhile, Claude will just do the thing one-shot or with a tiny bit of refinement. The gateway to knowledge and getting stuff done is the LLM. Google Search is a dinosaur. It feels like we're living a century into the future. Not even smartphones were this…
>It feels like we're living a century into the future.
The WALL-E chair-people future.
Re: How to setup a local coding agent on macOS
#105My biggest pet peeve with all these articles on local AI is the only thing they talk about is tokens per second. No one mentions the quality of the answers. No one. I don't mind waiting a little longer if the quality is better. Quickly serving me slop doesn't make it more useful. Are people really only looking at tokens per second?
Local model as such will give you "autocomplete on steroids" but it is not going to run away and implement cross project feature like frontier model in let's say Cursor. So there is no value in testing quality of answers, but there is value in testing token speed. You just have to have correct expectations.
Is autocomplete using LLMs really useful? Even with frontier models I found it to be about 50% right, I turned it of and prefer to use IntelliJ built-in, it is way more reliable.
For me local models is all about quality, and how to achieve that - e.g. by providing guardrails that test the job done.
Re: How to setup a local coding agent on macOS
#106Earlier quoted context omitted.
Yeah, if the future is "Claude, think for me" I'm happy to stay at the good old present.
https://en.wikipedia.org/wiki/Is_Google_Making_Us_Stupid%3F https://newsletter.pessimistsarchive.org/p/when-educators-mo... New decade, same old argument. It's not > "Claude, think for me" It's > "Claude, be my subordinate and get this done for me" Instead of complaining on the sidelines, I'm getting a shit ton of work done.
The argument was correct then (Google/social did make us more stupid) and correct now regarding AI. So not sure why pointing out it was said before is relevant. Except as an example of its prescience.
>"Claude, be my subordinate and get this done for me"
Since "this" is thinking, then the two formulations are equivalent.
>Instead of complaining on the sidelines, I'm getting a shit ton of work done.
Until you no longer have a job and are drowned in slop.
Re: How to setup a local coding agent on macOS
#107I assumed lmstudio is the obvious choice after ollama. Is there a reason lmstudio is not used widely ?
Why would anyone use Ollama at all (aside from obvious reasons one can look up online) - llama.cpp used directly, without this wrapper is faster.
Basically one has two real choices for local LLMs: llama.cpp (if single user) or vLLM (if multi-user/enterprise).
Re: How to setup a local coding agent on macOS
#108or you can just load up ollama, have it load a local model and point claude or opencode at it... is this article old? It's not. I'm not sure why he went through all the bother of llama.cpp
ollama is a wrapper on top of llama.cpp, and it makes llama.cpp slower, why use it?
Also Ollama has other issues (like forgetting what it really is - a wrapper).
Re: How to setup a local coding agent on macOS
#109For high Ram (unified), and relatively middling to lowish Tflops and bandwidth GB/s, usually MoEs are most hopeful. The current top-1 in the (iq, tok/s, @ context depth) ranks for me (M2 Max, 96gb) is DeepSeek-V4-Flash REAP25 https://github.com/ljubomirj/ds4/tree/reap-compact-support
DS4F dropping to unusable <10 tok/s only at 784K context (!!) makes a big difference.
Re: How to setup a local coding agent on macOS
#110FYI you can open Claude code in the terminal, point it at this article and just tell it to "do it", if you're feeling extra lazy
This is the way. I'm not Googling much of anything anymore. 9/10 times the information is awful, it's hard to parse out of whatever other spam it's surrounded by. Meanwhile, Claude will just do the thing one-shot or with a tiny bit of refinement. The gateway to knowledge and getting stuff done is the LLM. Google Search is a dinosaur. It feels like we're living a century into the future. Not even smartphones were this…
There’s no way this is not a paid comment, I see stuff like this everywhere in HN nowadays