Live data from Hacker News

Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

news.ycombinator.com

261–270 of 620 posts

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#261
But, guys, when you say Claude/ GPT models, do you stop to think what are these "models"?

One day I thought about how can GPT send thinking parts one after another with a markdown header summary of the thinking block itself. Just think about it.

As a matter of fact, think about these operations, api endpoints, observe their output.

These so called SOTA models are not what meets the eye, and are not at all comparable in the infra department to local models. There is crazy orchestration going on due to the scale of these operations. But also these hard constraints lead to innovation. Innovation nobody speaks about.

I wouldn't say we cannot catchup, but serving our local models through llama, vllm is just the A, B, C of it all. In reality I think what is needed is a replication of said orchestration which I hinted at above.

The SOTA models are a deep orchestration of multiple models operating together it isn't a single model. As such no single model ever will catchup to them until it replicates through training first and then maybe through model architecture this orchestration.

Finally, I would wager that the SOTA "models", as one of these models in this orchestration setup, as served for general consumption, are not so much more capable than qwen 3.6.

I am sure that if you change your perspective you will start noticing the scale of the "magic".

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#262
I think it is work to set up but I'm also learning a lot setting it up. Mainly using qwen/qwen3.6-35b-a3b mlx with my 48GB M4 MBP which leaves me just enough headroom for docker dev-container and other basics. I use LM Studio to run and am using it via VSCode. A big difference made the system prompt improving the tool integration (I asked GPT for guidance on that). Before that it was not making changes but regenerating code often messing up than helping.

I mostly run my MBP on low power even when it is plugged in to avoid the noise and heat. Full power maybe doubles speed but more than doubles power.

What can it do: Simple restructuring of pages. Where did it and other models fail: Splitting up Pinia store which GPT-5.4 did without fail. I think with more tuning, guidance for tool use and maybe some support tooling around it performance can increase further.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#263
post #25
post #4

Not “local” and not interactive coding but sharing since it might be helpful. I have 2x RTX Pro 6000 Blackwell running DeepSeek V4 Flash. I get 160 tok/s raw but it’s a reasoning model. For my use case, I have it auto-write code and another system auto-review the code. I occasionally use it with pi to write some code and it’s blazing fast but it’s mostly habit that keeps me with CC and Codex.

Have you measured your electricity consumption for this rig? I have to wonder how much it would cost you per month.

Here's a DeepSeek-V4-Flash benchmark on 2X RTX Pro 6000:

  - Prefill: ~10K tok/s
  - Decode: 190 | 375 | 980 tok/s (for 1 | 4 | 16 concurrent requests)
  - GPU power draw during benchmark: Average: 585W | Max: 849W | Limit: 1200W with undervolt. Idle PC is 125W.
I've asked it to calculate the following considering a realistic blend of cached prompts and decode for agentic dev scenario.

Electricity-only (@ USD $0.08/kWh)

  Usage          | IN price  | OUT price | Monthly cost
  Concurrency=1  | $0.040/M  | $0.080/M  | $8.65 to $38.88 (5% to 100% active)
  Concurrency=4  | $0.024/M  | $0.044/M  | up to $48.67 (cheaper per token but higher power draw)
Total cost of ownership over 3 years is electricity + USD $20K (pre-hike pricing). In a production scenario, how much would I have to charge my users to break even, aiming for 4 concurrent requests 24/7?

A) Breakeven API pricing (est. 2B IN + 1B OUT throughput/month):

                        IN price    OUT price
  Self-hosted           $0.121/M    $0.363/M
  OpenRouter (budget)   $0.098/M    $0.196/M
  OpenRouter (DeepSeek) $0.140/M    $0.280/M
B) Breakeven subscription (users active ~1.5h/day):

    1 user: $563/mo (oh, hai)
    25 users: $23/mo
    100 users: $6/mo

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#264
post #200

Earlier quoted context omitted.

Which Opus? They certainly outperform Claude 3 Opus. Anyhow, feel free to try them out head to head on OpenRouter. I'd love to see someone write up their results, of a modern local sized open source model vs. frontier models from ~a year ago, on something other than the standard benchmarks.

There's a guy on Youtube named Bijan Bowen who tests all the models (open and frontier) on a series of one/few shot programming exercises and has been for a long while now. You can pretty much watch him compare the results for any two models you're likely to be interested in. I'm not affiliated, I just like his style and have found it handy. I know it's not very rigorous, but it's good enough for me and I've found hi…

OK, it looks like he did a browser OS test with both Claude 4 Opus and Qwen 3.6 35B-A3B.

Claude 4 Opus: https://youtu.be/J7omabtqnBM?t=193

Qwen 3.6 35B A3B: https://youtu.be/gVU-DQeqkI0?t=215

Qwen 3.6 produced far more working functionality than Claude 4 Opus did.

Obviously, just one test of a single one-shot prompt of a silly toy OS, but yeah, this particular test shows Qwen 3.6 running locally dramatically outperforming Claude 4 Opus, which was a frontier model a year ago.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#265
post #58

Will the AI labs always make sure there is at least a years worth of differential? I guess the underlying business premise is that each new release has a step function change that prevents this kind of behaviour..

If the government is going to gate access to frontier models from here on out, even if new releases are a step function change… which they’re not… then it may be even more comparable to what’s available with a subscription.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#266
post #261

But, guys, when you say Claude/ GPT models, do you stop to think what are these "models"? One day I thought about how can GPT send thinking parts one after another with a markdown header summary of the thinking block itself. Just think about it. As a matter of fact, think about these operations, api endpoints, observe their output. These so called SOTA models are not what meets the eye, and are not at all comparable…

> The SOTA models are a deep orchestration of multiple models operating together it isn't a single mode

I don't understand, why does it make you think this is the case?

> how can GPT send thinking parts one after another with a markdown header summary of the thinking block itself

Can you give an example?

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#268
post #253

Earlier quoted context omitted.

I am forced to use Qwen 3.6 27b at work and found it next to useless. I might as well do all the work manually rather than having it implement another mess or get the debugging entirely wrong. It feels like anything less than Sonnet is just a waste of time, apart from use as a smarter search function. It also strikes me as strange that you would mention Codex for UI polish, as it's notoriously bad at UI, and far behi…

It might be good at analysis & review, writing documentation, git commits, etc--even if it's not good at coding. All the drudgery.

Bad AI written documentation and commits are not great, particularly when you work in a team.

I almost find it offensive when colleagues open a MR with an obvious slop description that's frequently inaccurate.

That said, I find AI useful for a lot of drudgery like resolving merge conflicts or splitting changes out into separate MRs.

Particularly with the latter I had issues with small models, they butchered the changes I wanted moved. Not even on the second attempt did GPT 5.4 mini manage to move 10-20 lines to another file without modifying them in the process.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#269
post #200

Earlier quoted context omitted.

Which Opus? They certainly outperform Claude 3 Opus. Anyhow, feel free to try them out head to head on OpenRouter. I'd love to see someone write up their results, of a modern local sized open source model vs. frontier models from ~a year ago, on something other than the standard benchmarks.

I’m normally comparing frontier open/cheap models against frontier closed source. I use deepseek/glm regularly, they’re fine and you can get real work done with them but it’s super obvious when you switch back to opus or even sonnet. A 3B active param MoE model is not comparable.

Yeah. I was pointing out that local 3b active models outperform frontier models from a year ago.

Will this trend continue? Who knows. Both the frontier and local model will probably continue to get better. Which one will hit the top of the S-curve first? Hard to say, really. But what you can do right now locally is better than what you could do a year ago on the frontier, and lots of people were already using it pretty heavily a year ago.

Hoever, November is when most folks agree that the frontier models got good enough for much of their work. Local models aren't quite there yet (where by "local" I mean "can run at reasonable speed and quant on a system less that $10,000 with today's RAM and GPU prices"). The biggest open weights models are getting there, but those require something like an 8x H100 server to reasonably run.

It's likely that there will always be a gap between frontier and local if you're comparing models at the same time, you can just do a lot more with terabytes of HBM than gigabytes of DDR. But will local models get good enough to be usable for useful work? For many folks, they already are.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#270
post #116

About 90% of my coding is on Qwen 3.6 27b and Open Code with some custom skills and Semble. It is NOT as smart as CC or Codex but its enough to get most of my work done. I didn't set out to replace CC and Codex (I have an RTX 6000 so the TPS is faster than I care about, but the RTX 6000 was originally for other work). I only tried this just to see how close you could get to a frontier model for coding as an experimen…

Same here, I use Qwen 3.6 27b (Q6 quant) with llama.cpp on an RTX 5090 using the pi agent exclusively now. The fact that it's local means that I never have to think about token pricing, quotas, time of day, or data sensitivity. I have limited the GPU from 600W to 450W which means the system stays whisper quiet during inference. I have become so "lazy" (in a good way), so far that I've started using the model for lots…

What context length and kv cache quant (if any) are you using? And MTP?
Post reply on HN