I think local LLM and laptop is not really compatible, for anything useful. You're gonna want a bigger box and have your laptop connect to that.
Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop
11–20 of 196 posts
Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop
#12Not my build and not coding, but I've seen some experimental builds (oss 20b on a 32gb mac mini) with Kiwix integration to make what is essentially a highly capable local private search engine.
Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop
#13llama.cpp + Qwen3-4B running on older PC with AMD Radeon GPU (Vulcan). Users connect via web UI. Usually around 30 tokens/sec. Usable.
Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop
#14On a side note I really thing latency is still important. Is there some benefit in choosing location for where you get your responses from? Like with Openrouter f.e.
Also I could think that a local model just for autocomplete could help reducing latency for completion suggestions.
Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop
#15I passed on the machine, but we set up gpt-oss-120b on a 128GB RAM Macbook pro and it is shockingly usable. Personally, I could imagine myself using that instead of OpenAI's web interface. The Ollama UI has web search working, too, so you don't have to worry about the model knowing the latest and greatest about every software package. Maybe one day I'll get the right drivers to run a local model on my Linux machine w…
LM Studio also works well on Mac.
Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop
#16I am here to hear from folks running LLM on Framework desktop (128GB). Is it usable for agentic coding ?
Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop
#17llama.cpp + Qwen3-4B running on older PC with AMD Radeon GPU (Vulcan). Users connect via web UI. Usually around 30 tokens/sec. Usable.
What do they use it for? It's a very small model
Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop
#18The M2/3/4 Max CPUs in a Mac Studio or Macbook Pro when paired with enough ram are quite capable.
In more cases than expected, the M1/M2 Ultras are still quite capable, especially performance power per watt of electricity, as well as ability to serve one user.
The Mac Studio has better bang for the buck than the laptop for computational power to price.
Depending on your needs, the M5's might be worth waiting for, but M2 Max onward are quite capable with enough ram. Even the M1 Max continues to be a workhorse.
Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop
#19Rtx 3090 24gb. Pretty affordable.
Gos-oss:20b and qwen3 coder/instruct, devstrall are my usual.
Ps. Definitely check out open-web ui.
Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop
#20I use the abliterated and uncensored models to generate smut. SwarmUI to generate porn. I can only get a few tokens/s on my machine so not fast enough for quick back and forth stuff.