Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

news.ycombinator.com

Post reply on HN
Print view View original

91–100 of 620 posts

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#91

Post by Rastonbury »

jrm4 wrote: post #45
codinhood wrote: post #26

I don't think you're going to get many "true" answers to this. The opportunity cost of not using the latest and best models is just too much right now. Every month I research this and come to the same conclusion: the time, effort, and cost required to get local models (and the coding tools around them) to perform even close to Claude Code with sonnet/opus just not worth it right now. If it was, it would be distributi…

But you're pretty much measuring opportunity cost in tokens per second, no? I think it strongly remains to be seen whether e.g. tokens per second (multiplied or whatever by percieved quality of private model) actually means "better or more useful output." I strongly suspect it does not. (though I also strongly suspect this will be very difficult to measure because the incentive to lie about metrics here will be so st…

I think they are referring to the opportunity cost of time saved on doing things a local model cannot do or fixing it's mistakes against the cost of a subscription

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#92

Post by akersten »

arjie wrote: post #4

Not “local” and not interactive coding but sharing since it might be helpful. I have 2x RTX Pro 6000 Blackwell running DeepSeek V4 Flash. I get 160 tok/s raw but it’s a reasoning model. For my use case, I have it auto-write code and another system auto-review the code. I occasionally use it with pi to write some code and it’s blazing fast but it’s mostly habit that keeps me with CC and Codex.

> I have 2x RTX Pro 6000 Blackwell

Where did you find/order these? All the sites I can find are either out of stock, only sell to businesses, or are otherwise sketchy...

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#94

Post by mirekrusin »

gonzalohm wrote: post #84
horsawlarway wrote: post #49

For personal use, yes. I replaced a $100/m subscription to claude in favor of running pi harness pointed at unsloth studio, using both qwen (unsloth/Qwen3.6-35B-A3B-MTP-GGUF) and gemma (unsloth/gemma-4-26B-A4B-it-GGUF) models, depending on my mood. I have a machine I built about 5 years ago with dual RTX3090s in it (I was going to build a new gaming machine anyways, and the llama release had just dropped so I tacked…

Did you double the tokens per second by adding a second GPU or was the increase significantly less?

You’re adding extra gpu for more vram, not speed.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#95

Post by SimianSci »

Razengan wrote: post #12

Related: Are there any viable distributed AI models? Like how we've had SETI at Home, Folding at Home, BitTorrent etc. People are clearly willing to donate their computer resources to distributed projects. Maybe in a dAI network anyone could submit content for training on, and each user running a "node" could have their own custom private conditions on which type of content to accept for training or inference. Like s…

This is unlikely to happen in any meaningful fashion for quite some time.

(TLDR; Distributed compute for models will require hardware at a level only really possible with data-centers at the moment.)

Token generation operates at such a scale to demand enough from a single GPU as it will often saturate the bandwidth capabilities of consumer grade interconnects like PCIe. Which fundamentally implies that distributing a model's compute across vast distances is too much of a challenge without significant infrastructure.

To give an example, When we split a model's compute between two seperate cards on a single workstation, this doesnt mean we end up with 2x the compute bandwidth for a model. Instead the increase becomes something small like 20% depending on model, because the inconnects (PCIe on consumer hardware) will quickly become so saturated with data being copied between the two GPUs so as to become a bottleneck. And remember that this is something that happens locally with PCIe, which (depending on generation) will cap out at around 20-35 GB/s depending on the generation of motherboard.

Model performance is very much tied to having the fastest and highest bandwidth single card available so as to keep data transfer operations to a minimum as the sheer volume of data necessary for the model to run is immense. I simply cant imagine how slow and unusable a model would be if the copy operations necessary for its compute needed to be performed over unreliable network speeds where there will be significant performance loss as network speeds are not reliably distributed across the globe, and their unreliable nature would demand increased overhead due to data verification.

The dream of distributed AI is a ways off.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#96

Post by lysace »

tobyhinloopen wrote: post #47
lysace wrote: post #31

Earlier quoted context omitted.

Got a similar result (my RTX 4070 only has 12 GB). I'm curious about whether 24/32 GB meaningfully improves this enough to make it useful.

Try it on RAM and CPU. It’s slower but you can run them.

Good idea for evaluating the models, thanks.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#97

Post by hoherd »

dheera wrote: post #70
pierotofy wrote: post #36

Yes. Llama.cpp + Qwen3.6-35b (MTP) + OpenCode is quite capable and runs on a single RTX 3090 and is faster than most cloud models. Quality is like running edge models from 8-12 months ago. Setup details at https://github.com/pierotofy/LocalCodingLLM/

Am I doing something wrong or has ollama become shittified? I'm looking at https://ollama.com/search and the top few models like kimi-k2.7-code say "cloud" and I can't seem to ollama pull them. I thought the whole POINT of ollama was not-cloud?

I experienced the same situation a month or two ago. One of my friends sent me this article that was illuminating. https://sleepingrobots.com/dreams/stop-using-ollama/

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#98

Post by agup792 »

horsawlarway wrote: post #49

For personal use, yes. I replaced a $100/m subscription to claude in favor of running pi harness pointed at unsloth studio, using both qwen (unsloth/Qwen3.6-35B-A3B-MTP-GGUF) and gemma (unsloth/gemma-4-26B-A4B-it-GGUF) models, depending on my mood. I have a machine I built about 5 years ago with dual RTX3090s in it (I was going to build a new gaming machine anyways, and the llama release had just dropped so I tacked…

That sounds amazing. If I had some GPUs sitting around, I would totally do it. Sounds expensive to do it otherwise though.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#99

Post by lokar »

jacobgold wrote: post #75
pierotofy wrote: post #67

Earlier quoted context omitted.

I use it for work.

That's cool if you prefer it, but it is hard to imagine it being a strictly rational choice when much better quality is available at a price that is small relative to the cost of an employee. Or is there something specific about your use-case?

Won’t it depend on what you use it for? A less capable system might be fine for boilerplate, moderate re-factoring, etc. Not everyone is building whole features in one go.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#100

Post by rootlocus »

horsawlarway wrote: post #49

For personal use, yes. I replaced a $100/m subscription to claude in favor of running pi harness pointed at unsloth studio, using both qwen (unsloth/Qwen3.6-35B-A3B-MTP-GGUF) and gemma (unsloth/gemma-4-26B-A4B-it-GGUF) models, depending on my mood. I have a machine I built about 5 years ago with dual RTX3090s in it (I was going to build a new gaming machine anyways, and the llama release had just dropped so I tacked…

2x RTX3090 are around $4400. Without any electricity costs or other parts, that's 3.6 years of $100/m claude.
Post reply on HN