Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

news.ycombinator.com

Post reply on HN
Print view View original

441–450 of 620 posts

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#441

Post by user43928 »

what wrote: post #351
user43928 wrote: post #246

Earlier quoted context omitted.

My experience with smaller models, in this case specifically GPT 5.4 Mini, is that they cannot two-shot moving a 10-20 line code change to another file without modifying it and introducing bugs. I did not expect perfect reliability, but I thought they could at least get it right on the second attempt once you point out the difference. No such luck, it confidently tells you that now the code is the same, with yet anot…

Is it not faster to just do that move yourself instead of asking the clanker to do it?

Typing "Create a branch for X and open a separate MR" is faster than me manually creating a branch, selecting and copying changes there, and then opening a MR.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#442

Post by Iolaum »

lambda wrote: post #155
Greenpants wrote: post #112

I have! I care about data privacy and LLMs being free. I'm using the Pi coding harness but containerized and sandboxed, to make sure it's running completely offline. On my Mac Studio with 128GB RAM (or MacBook with 36GB RAM) I'm using Qwen3.6 35b, with only 3b active parameters so that it runs really fast. I've done a complete redesign for my website's homepage and blog with Django + Wagtail. The latter is interestin…

This is very similar to my setup. Pi in a container (I do let it have network access, just no access to creds or anything, only the one directory that I'm working on at the time and my ~/.pi directory), talking to llama.cpp in another container. I'm on a Strix Halo 128 GiB unified memory laptop. I've never used the frontier models in earnest, I don't believe in using proprietary tools for my programming, so I can't r…

Haven't used for actual coding but was testing locally - for example running some swebench instances - whether qwen-3.6-35b-a3b@Q8 was better than qwen-3.5-122b-a10b@Q4. With MTP the former runs at around 55t/s and the latter at around 30t/s meaning the latter is also usable. It looked like qwen-3.5-122b-a10b@Q4 performed a bit better.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#443

Post by toyg »

overgard wrote: post #214
tripleee wrote: post #184

Earlier quoted context omitted.

Christ GPU prices have gotten crazy How do AMD cards perform with LLMs? A 9070 is sold for ~$600 and has 16GB VRAM

In my personal experience, I wouldn't bother with 16GB cards for coding -- the useful models are _slightly_ too large to work at any reasonable speed

That's not my experience, and the trajectory is good anyway - what doesn't work perfectly today will be just fine in a few months.

In a quickly moving field, it's amazing how much money one can save by overcoming FOMO and not living on the bleeding edge. It's like waiting for Steam sales, the games will be just as good.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#444

Post by heipei »

amarshall wrote: post #270
heipei wrote: post #116

Earlier quoted context omitted.

Same here, I use Qwen 3.6 27b (Q6 quant) with llama.cpp on an RTX 5090 using the pi agent exclusively now. The fact that it's local means that I never have to think about token pricing, quotas, time of day, or data sensitivity. I have limited the GPU from 600W to 450W which means the system stays whisper quiet during inference. I have become so "lazy" (in a good way), so far that I've started using the model for lots…

What context length and kv cache quant (if any) are you using? And MTP?

No KV cache quant, context length 50% of original, MTP absolutely. These are the relevant cmdline attributes. Getting around 100t/s with this setup, even when watt-limited to 450W.

  --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.00 --presence-penalty 0 --metrics --jinja --chat-template-file chat_template.jinja --chat-template-kwargs '{"preserve_thinking": true}' --spec-type draft-mtp --spec-draft-n-max 2 --spec-draft-p-min 0.75 -ngl 99 -c 131072 -fa on -np 1 -hf unsloth/Qwen3.6-27B-MTP-GGUF:Q6_K

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#447

Post by bob1029 »

codinhood wrote: post #26

I don't think you're going to get many "true" answers to this. The opportunity cost of not using the latest and best models is just too much right now. Every month I research this and come to the same conclusion: the time, effort, and cost required to get local models (and the coding tools around them) to perform even close to Claude Code with sonnet/opus just not worth it right now. If it was, it would be distributi…

I've got a machine in a corner collecting dust that cost me $12k to build 2 years ago. It runs fine but it's wildly impractical to use as a daily driver (loud/hot). I keep it as a reminder to not do this again.

At my current pace it would take me until sometime late 2030 to spend the same amount in gpt5.5 tokens.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#449

Post by Kostic »

fitzn wrote: post #354
Kostic wrote: post #55

For personal needs I connected VSCode with llama.cpp running Qwen 3.6 27B or Gemma 4 31B and it's good enough to cancel my cloud subscription. Qwen running on my 1st GPU at q4@176k context from 70 to 50 tok/s with MTP, pretty good for coding. Gemma on the other hand is using both GPUs, running q8@64k context, doing document sentiment analysis, summarization, proofreading and translating, at consistent 25 tok/s. Somew…

What extension do you use in vscode to connect it to local llama.cpp? Or do you auth with github copilot and then point to localhost? Or something else?

Auth with Github Copilot and then point it to localhost[0]. Hopefully the auth to Copilot requirement will be dropped for local models at some point. Would love to use a fully open stack (VSCodium and everything) in the future. My config:

``` [ { "name": "http://127.0.0.1:8888/v1", "vendor": "customendpoint", "apiKey": "llama.cpp", "models": [ { "id": "gemma4-31b", "name": "Gemma 4 31B", "url": "http://127.0.0.1:8888/v1/chat/completions", "toolCalling": true, "vision": true, "maxInputTokens": 65536, "maxOutputTokens": 8192 }, { "id": "qwen3.6-27b", "name": "Qwen 3.6 27B", "url": "http://127.0.0.1:8888/v1/chat/completions", "toolCalling": true, "vision": true, "maxInputTokens": 180224, "maxOutputTokens": 8192 } ] } ] ```

[0] https://code.visualstudio.com/blogs/2025/10/22/bring-your-ow...

Post reply on HN