Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

news.ycombinator.com

Post reply on HN
Print view View original

421–430 of 620 posts

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#421

Post by kristopolous »

Greenpants wrote: post #112

I have! I care about data privacy and LLMs being free. I'm using the Pi coding harness but containerized and sandboxed, to make sure it's running completely offline. On my Mac Studio with 128GB RAM (or MacBook with 36GB RAM) I'm using Qwen3.6 35b, with only 3b active parameters so that it runs really fast. I've done a complete redesign for my website's homepage and blog with Django + Wagtail. The latter is interestin…

I've got a tool that sits in between the harness and inference engine called petsitter. It is a middleman validator to avoid just these kinds of issues. You can stack the fixes as needed (they're called tricks in the petsitter parlance)

It's what I use. Fixes the problem

https://github.com/day50-dev/petsitter

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#422

Post by system2 »

ColonelPhantom wrote: post #311
system2 wrote: post #21

Until I can buy an 80GB VRAM GPU, I won't attempt to do it. A local LLM is always missing something that needs a bigger model.

Which model class requires an 80 GB VRAM GPU? From my perspective, popular models seem to be either in the ~30B range (Qwen3.6, Gemma 4), while the larger models (MiniMax, MiMo, StepFun, Deepseek) are in the multiple hundreds of billions parameters, for which 80 GB is simply too small. You can just about reach the lower end of the latter category with a 128GB machine like a DGX Spark, Framework Desktop, or M5 Max, th…

Video models.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#423

Post by stymaar »

thefroh wrote: post #403
lambda wrote: post #188

Earlier quoted context omitted.

I use Vulkan mostly instead of ROCm. Vulkan is actually a bit faster, paradoxically. I do switch out and try them both out, and it's not a huge difference, but I've been mostly saying on Vulkan. The re-processing context every turn problem is definitely something I've hit. Some of the causes have been solved upstream in llama.cpp; make sure you're up to date. But another cause of the issue that has a big effect is th…

I'm a little surprised that preserve_thinking would matter here for cache purposes. for actual capabilities/intelligence, yes, I'd imagine it helps to have past reasoning traces in multi-turn setups. but for caching, all you are doing is leaving off a fraction of the most recent assistant message generation, which will have little/no impact on cache hit rate.

> all you are doing is leaving off a fraction of the most recent assistant message generation

True, but not a tiny fraction, qwen is very verbose in its thinking traces. And it basically means that for every (nonthinking) generated token you have to compute the KV twice (once as tg, the second one as pp).

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#424

Post by luipugs »

femto113 wrote: post #367
fjdjshsh wrote: post #343

Earlier quoted context omitted.

>I'm still a AI skeptic What does this mean in June 2026 wrt coding? To me it sounds like being a "rice cooker skeptic". Some people don't like using rice cookers, some do.

For me the distinction is that your rice only needs to be edible once, while your code may need to last for decades. Using AI to code anything I could comfortably throw away if needed is a lot less fraught than letting it make choices that I and anybody who inherits the code is gonna have to live with, especially if by outsourcing those choices I reduce my understanding of the implications of those choices.

Don't you read through all the output of the agent before committing them?

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#425

Post by kristopolous »

codinhood wrote: post #26

I don't think you're going to get many "true" answers to this. The opportunity cost of not using the latest and best models is just too much right now. Every month I research this and come to the same conclusion: the time, effort, and cost required to get local models (and the coding tools around them) to perform even close to Claude Code with sonnet/opus just not worth it right now. If it was, it would be distributi…

that's super contextually dependent. I use them just as essentially a decompress of what I already know that I'm doing. I legitimately use 4B models just fine. I've got a large number of tools that make this entirely feasible and a daily driver for me (like https://github.com/day50-dev/llm-manpage-tool) ...

It's not really a bitter lesson here, I can scale those 4B models easier than someone can scale their 1000B models.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#426

Post by davkan »

augusto-moura wrote: post #141
freetonik wrote: post #114

Earlier quoted context omitted.

That's also years of top tier PC gaming, if you're into that.

2x RTX3090 is extremely overkill for gaming, you can run any released game on earth on ultra for much less

There is currently no gpu in production that can max out the largest and fastest displays in graphically demanding games. We have monitors that are the equivalent of two 4k monitors side by side and run at 240hz. I have a 5080 and have to turn down settings to get 60fps in cyberpunk.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#427

Post by utopiah »

rvnx wrote: post #167
dabinat wrote: post #35

There’s evidence that combining models can achieve frontier-level performance (e.g. OpenRouter Fusion). I’m wondering if that’s the more realistic option: combine Opus with a local model to save on token costs.

I start to believe that adding more and more and more and more and more thinking tokens is the hack that works (this is what gave birth to Fable)

Why would you not think that?

It seems pretty intuitive that pouring more resources into a problem (more GPU, bigger GPUs with more VRAM, bigger datasets, better curated datasets, more efficient ways to train, more efficient way to run inference, etc) then running the result for a longer time, with more layers of verification (running in VMs, model fusion comparing multiple models, having harnesses with testing) will at least lead to marginally better results.

Is it worth it and at what pace will it keep on improving are different questions but I have little doubt that if the industry keep on pouring resources, sure more "works".

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#428

Post by bandrami »

hparadiz wrote: post #187
Greenpants wrote: post #112

I have! I care about data privacy and LLMs being free. I'm using the Pi coding harness but containerized and sandboxed, to make sure it's running completely offline. On my Mac Studio with 128GB RAM (or MacBook with 36GB RAM) I'm using Qwen3.6 35b, with only 3b active parameters so that it runs really fast. I've done a complete redesign for my website's homepage and blog with Django + Wagtail. The latter is interestin…

I am right there with you. Mind-boggling. It's a indistinguishable from magic technology!! I tried running some basic tasks through Qwen with Opencode on a 10 year old dual Xeon server for shits and giggles. I gave it a simple task like "use ffprobe first but convert this webm to mp4" and it was able to complete the task with zero network calls outside my network. On 10 year old hardware. It took about 3 minutes to c…

> You're gonna be googling the CLI switches for at least 10 minutes

So there's this really amazing program called "man"

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#429

Post by secult »

luipugs wrote: post #424
femto113 wrote: post #367

Earlier quoted context omitted.

For me the distinction is that your rice only needs to be edible once, while your code may need to last for decades. Using AI to code anything I could comfortably throw away if needed is a lot less fraught than letting it make choices that I and anybody who inherits the code is gonna have to live with, especially if by outsourcing those choices I reduce my understanding of the implications of those choices.

Don't you read through all the output of the agent before committing them?

That's not the way how human brain works.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#430

Post by p0w3n3d »

Greenpants wrote: post #112

I have! I care about data privacy and LLMs being free. I'm using the Pi coding harness but containerized and sandboxed, to make sure it's running completely offline. On my Mac Studio with 128GB RAM (or MacBook with 36GB RAM) I'm using Qwen3.6 35b, with only 3b active parameters so that it runs really fast. I've done a complete redesign for my website's homepage and blog with Django + Wagtail. The latter is interestin…

which coding agent are you using?
Post reply on HN