Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

news.ycombinator.com

Post reply on HN
Print view View original

431–440 of 620 posts

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#431

Post by deaux »

sbrother wrote: post #81
jacobgold wrote: post #62

Earlier quoted context omitted.

"Quality is like running edge models from 8-12 months ago." That sounds great for hobbyists but IMHO it wasn't until Opus 4.6 was released six months go (Dec 25, 2025) that we had a model good enough for professionals to use as a primary driver of their coding agents. That seems to be the threshold worth aiming for.

I strongly agree on that being the release where these tools got good enough to substantially speed up my professional work. I have to admit I was super skeptical of AI coding until then.

Your skepticism led you to underrate the usefulness until then. Those who have been using agentic coding for the last 2 years can tell you Opus 4.6 was not a step change in quality, it was mostly a step change in the Overton Window and narrative.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#432

Post by havfo »

chakspak wrote: post #173
lambda wrote: post #155

Earlier quoted context omitted.

This is very similar to my setup. Pi in a container (I do let it have network access, just no access to creds or anything, only the one directory that I'm working on at the time and my ~/.pi directory), talking to llama.cpp in another container. I'm on a Strix Halo 128 GiB unified memory laptop. I've never used the frontier models in earnest, I don't believe in using proprietary tools for my programming, so I can't r…

Hopefully this isn't off-topic, but your setup sounds just like mine, Strix Halo and (I'm assuming) llama.cpp on ROCm, and I'm finding that the Qwen hybrid models don't handle prompt caching and instead re-process the context in full on every turn. I'm wondering if you were able to solve this and how?

I was able to solve this for my setup, 7900XTX and llama.cpp on ROCM in the oh-my-pi fork of pi.dev harness. I documented my setup on github, check under my username/omp-config, but the important thing is making sure the context is strictly append-only, and starting llama.cpp with

  --chat-template-kwargs '{"preserve_thinking":true}'

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#433

Post by ljosifov »

Not replaced but supplemented. For off-line coding current setup is pi + ds4-server + DeepSeek-V4-Flash REAP25 (on M2 Max 96gb). For simpler programming related (e.g. text2sql) as well as synthetic data generation, current best for me is llama.cpp + Gemma-4-26B-A4B (on gpu 7900xtx 24gb; sometimes nemotron-cascade-2-30b-a3b for 1M context). That and (dabbling now) auto-research uses lots of tokens. Used to get paused running out of token quotas all the time. The 1st local model I found somewhat useful to me was glm-4.7-flash, and it's gotten way better since. Recently between OpenCode Go choice of models at many price points, and DeepSeek-V4 dropping the IQ/$$$ by multiples, have become less reliant on local llms for this auxiliary work. Claude I use but with Zai GLM-5.2 subscription. And maintain GPT subscription for quality models.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#434

Post by l332mn »

horsawlarway wrote: post #49

For personal use, yes. I replaced a $100/m subscription to claude in favor of running pi harness pointed at unsloth studio, using both qwen (unsloth/Qwen3.6-35B-A3B-MTP-GGUF) and gemma (unsloth/gemma-4-26B-A4B-it-GGUF) models, depending on my mood. I have a machine I built about 5 years ago with dual RTX3090s in it (I was going to build a new gaming machine anyways, and the llama release had just dropped so I tacked…

Mind sharing your setup? I also have dual 3090s, but getting nowhere close to 300k context limits with 4 bit quantized models at that size (using vllm).

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#435

Post by Greenpants »

awllau wrote: post #360
Greenpants wrote: post #112

I have! I care about data privacy and LLMs being free. I'm using the Pi coding harness but containerized and sandboxed, to make sure it's running completely offline. On my Mac Studio with 128GB RAM (or MacBook with 36GB RAM) I'm using Qwen3.6 35b, with only 3b active parameters so that it runs really fast. I've done a complete redesign for my website's homepage and blog with Django + Wagtail. The latter is interestin…

Based on your explanation, it doesn't sound feasible for me, a complete non-engineer, to switch to fully offline? I do a lot of back and forth discussion with LLMs as someone who reads and writes 0 code.

I'm afraid I'd have to agree. That is, unless you have 512GB+ RAM sitting on a shelf and run the much larger SOTA-comparable local models.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#436

Post by big-chungus4 »

I can run Qwen3.6-35B-A3B at 20 TPS on my laptop with RTX 5070 Ti, with partial offloading to RAM. But the most I do is mess with it when I'm bored. I do coding by hand, but I often run autoresearch loops using free models, right now it's MiMo code. Autoresearch often requires my GPU, so it wouldn't be feasible to do when all of my GPU is used up by a local model. For mundane tasks like extracting and formatting specific structured text, I use Gemini in Google search

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#439

Post by Gigachad »

phyzix5761 wrote: post #363
codinhood wrote: post #26

I don't think you're going to get many "true" answers to this. The opportunity cost of not using the latest and best models is just too much right now. Every month I research this and come to the same conclusion: the time, effort, and cost required to get local models (and the coding tools around them) to perform even close to Claude Code with sonnet/opus just not worth it right now. If it was, it would be distributi…

The opportunity cost to who? Its getting super expensive for businesses and engineers across the board to pay for frontier models.

The cost of the hardware to run local models is still massively more expensive than the subscriptions while offering worse models.

Eventually I think it will even out but right now the hosted stuff is very subsidised.

Post reply on HN