Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop

#1

Post by threeturn » Fri, Oct 31, 2025, 1:39 PM UTC

Dear Hackers, I’m interested in your real-world workflows for using open-source LLMs and open-source coding assistants on your laptop (not just cloud/enterprise SaaS). Specifically:

Which model(s) are you running (e.g., Ollama, LM Studio, or others) and which open-source coding assistant/integration (for example, a VS Code plugin) you’re using?

What laptop hardware do you have (CPU, GPU/NPU, memory, whether discrete GPU or integrated, OS) and how it performs for your workflow?

What kinds of tasks you use it for (code completion, refactoring, debugging, code review) and how reliable it is (what works well / where it falls short).

I'm conducting my own investigation, which I will be happy to share as well when over.

Thanks! Andrea.

Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop

#2

Post by lreeves » Fri, Oct 31, 2025, 3:13 PM UTC

I sometimes still code with a local LLM but can't imagine doing it on a laptop. I have a server that has GPUs and runs llama.cpp behind llama-swap (letting me switch between models quickly). The best local coding setup I've been able to do so far is using Aider with gpt-oss-120b.

I guess you could get a Ryzen AI Max+ with 128GB RAM to try and do that locally but non-nVidia hardware is incredibly slow for coding usage since the prompts become very large and take exponentially longer but gpt-oss is a sparse model so maybe it won't be that bad.

Also just to point it out, if you use OpenRouter with things like Aider or roocode or whatever you can also flag your account to only use providers with a zero-data retention policy if you are truly concerned about anyone training on your source code. GPT5 and Claude are infinitely better, faster and cheaper than anything I can do locally and I have a monster setup.

Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop

#3

Post by ge96 » Fri, Oct 31, 2025, 3:30 PM UTC

I don't, although I'm not a puritan eg. I'll use the AI summary that shows first in browsers

Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop

#4

Post by baby_souffle » Fri, Oct 31, 2025, 3:33 PM UTC

Good quality still needs more power than what a laptop can do. The local llama subreddit has a lot of people doing well with local rigs, but they are absolutely not laptop size.

Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop

#5

Post by juujian » Fri, Oct 31, 2025, 3:34 PM UTC

I passed on the machine, but we set up gpt-oss-120b on a 128GB RAM Macbook pro and it is shockingly usable. Personally, I could imagine myself using that instead of OpenAI's web interface. The Ollama UI has web search working, too, so you don't have to worry about the model knowing the latest and greatest about every software package. Maybe one day I'll get the right drivers to run a local model on my Linux machine with AMD's NPU, too, but AMD has been really slow on this.

Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop

#6

Post by sho » Fri, Oct 31, 2025, 3:35 PM UTC

Real-world workflows? I'm all for local LLM, tinker with it all the time, but for productive coding use no local LLM approaches cloud and it's not even close. There's no magic trick or combination of pieces, it just turns out that a quarter million dollars worth of H200s is just much, much better than anything a normal person could possibly deploy at home.

Give it time, we'll get there, but not anytime soon.

Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop

#7

Post by bravetraveler » Fri, Oct 31, 2025, 3:35 PM UTC

I'm more local than anything, I guess. A Framework Desktop off in another room. 96G set aside for VRAM though I barely use it.

Kept it simple: ollama, whatever the latest model is in fashion [when I'm looking]. Feel silly to name any one in particular, I make them compete. I usually don't bother: I know the docs I need.

Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop

#8

Post by firefax » Fri, Oct 31, 2025, 3:36 PM UTC

I've been using Ollama, Gemma3:12b is about all my little air can handle.

If anyone has suggestions on other models, as an experiment I tried asking it to design me a new latex resumé and it struggled for two hours with the request to put my name prominently at the top in a grey box with my email and phone number beside it.

Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop

#9

Post by reactordev » Fri, Oct 31, 2025, 3:37 PM UTC

I use LM Studio with GGUF models running on either my Apple MacBook Air M1 (it’s, ok…) or my Alienware x17 R2 with an RTX 3080 on a Core i9 (runs like autocomplete) in VS Code using Continue.dev

My only complaint is agent mode needs good token gen so I only go agent mode on the RTX machine.

I grew up on 9600baud so I’m cool with watching the text crawl.

Re: Ask HN: Who uses open LLMs and coding assistants locally? Share setup and laptop

#10

Post by starik36 » Fri, Oct 31, 2025, 3:40 PM UTC

sho wrote: post #6

Real-world workflows? I'm all for local LLM, tinker with it all the time, but for productive coding use no local LLM approaches cloud and it's not even close. There's no magic trick or combination of pieces, it just turns out that a quarter million dollars worth of H200s is just much, much better than anything a normal person could possibly deploy at home. Give it time, we'll get there, but not anytime soon.

You are right. This is the current situation. Plus the downside is that your laptop heats up like a furnace if you use the local LLM a lot.