Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

news.ycombinator.com

Post reply on HN
Print view View original

491–500 of 620 posts

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#491

Post by girvo »

ttoinou wrote: post #460
girvo wrote: post #293

Earlier quoted context omitted.

Right. Tokens/s decode isn't the most important thing to me: wall clock time for task completion is. And tracking all of that, on my GB10-based Asus box, Step 3.7 Flash at IQ4_XS beats Qwen 3.6 27B despite the latter having MTP, on all of my actual coding task evaluations in real codebases. Qwen seems better at one-shotting things based on vague prompts to an acceptable degree, but thats literally not what I use thes…

I tried Step 3.7 Flash on my mac 128GB and it seemed very dumb. antirez ds4 flash is much better !

It isn’t though, I’ve run both through a bunch of coding evals. You nearly certainly didn’t have the right sampling parameters or quantised the KV cache?

Ds4 is impressive for what it is, but it loops and over thinks even more, burning massive wall clock time to not even get great outcomes. It’s also limited to a slow speed on my Spark

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#493

Post by antirez »

mtone wrote: post #263
leptons wrote: post #25

Earlier quoted context omitted.

Have you measured your electricity consumption for this rig? I have to wonder how much it would cost you per month.

Here's a DeepSeek-V4-Flash benchmark on 2X RTX Pro 6000: - Prefill: ~10K tok/s - Decode: 190 | 375 | 980 tok/s (for 1 | 4 | 16 concurrent requests) - GPU power draw during benchmark: Average: 585W | Max: 849W | Limit: 1200W with undervolt. Idle PC is 125W. I've asked it to calculate the following considering a realistic blend of cached prompts and decode for agentic dev scenario. Electricity-only (@ USD $0.08/kWh) Us…

Interestingly if we assume 16 concurrent users, prefill drops to 600 t/s and generation to 61 t/s, and this starts to be dangerously near to M5 Max 35 t/s generation and 400 t/s prefill you get with DwarfStar in your own laptop (that you use for many other things) that costs ~6500 usd/eur.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#494

Post by _zoltan_ »

monooso wrote: post #487
_zoltan_ wrote: post #418

Earlier quoted context omitted.

We're discussing home use. You can. You just don't want to. Huge difference.

> We're discussing home use. You may be, but the topic of discussion is whether anyone is using a local model as their main coding tool.

for corporate use it's a mistake not to use a frontier model.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#495

Post by Applejinx »

user43928 wrote: post #246
sosodev wrote: post #32

The problem with this question is that it encompasses a huge spectrum of capabilities and expectations. If you can only run an 8B model and expect it to be good at vibe coding / one shotting things you're going to have a bad time. If you're able to run a model on the scale of ~30B, you can find that with a reasonably scoped and well defined task they do very well. I've found both Gemma4-31B and Qwen3.6-27B to be the…

My experience with smaller models, in this case specifically GPT 5.4 Mini, is that they cannot two-shot moving a 10-20 line code change to another file without modifying it and introducing bugs. I did not expect perfect reliability, but I thought they could at least get it right on the second attempt once you point out the difference. No such luck, it confidently tells you that now the code is the same, with yet anot…

Rather than 'smarter search or autocomplete', maybe the better analogy is 'flexible information lookup about coding that's more responsive to search terms'? I see it not as an intelligence but as a wildly, spectacularly compressed knowledge base. You're trying to get search results that encompass almost any possible thing you could ask for, but rather than drawing from some textbook you're drawing from a distilled combination of ALL textbooks and everything web-scrapable since before the dotcom days.

Of course this doesn't produce a useful person who always makes right choices, but isn't it interesting that you can compress that heavily and draw results out in such a casual way? Seems this remains relevant.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#496

Post by gwerbin »

ClikeX wrote: post #470
ojr wrote: post #315

Earlier quoted context omitted.

I can use Gemini 3 Flash with the harness I built for around 8 years and still not exceed the cost of a Mac Studio with 128GB, the price for privacy is very high. Agentic flows that get stuck can be worked around but I prefer developer velocity.

> the price for privacy is very high Not sure if you intended this to be this philosophical, but this is basically the slogan for modern life now.

Yeah but the price for, say, private email is a lot less.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#497

Post by ttoinou »

girvo wrote: post #491
ttoinou wrote: post #460

Earlier quoted context omitted.

I tried Step 3.7 Flash on my mac 128GB and it seemed very dumb. antirez ds4 flash is much better !

It isn’t though, I’ve run both through a bunch of coding evals. You nearly certainly didn’t have the right sampling parameters or quantised the KV cache? Ds4 is impressive for what it is, but it loops and over thinks even more, burning massive wall clock time to not even get great outcomes. It’s also limited to a slow speed on my Spark

I tried a bunch of stuff with step 3.5 and step 3.7 maybe not as much as you. Could you tell me what parameters and launched you’re using ? Antirez ds4 flash q2-q4 works almost out of the box for me

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#498

Post by gwerbin »

girvo wrote: post #293
electronsoup wrote: post #189

Earlier quoted context omitted.

> It gets into loops quite often, and surprisingly often gets the edit tool call wrong I find that running better quantization, like Q8 tend to prevent this even though its a bit slower to run, it saves overall time with less churn Using 3.6-27b is even slower again than 3.6-35b, but I find the accuracy really pays off

Right. Tokens/s decode isn't the most important thing to me: wall clock time for task completion is. And tracking all of that, on my GB10-based Asus box, Step 3.7 Flash at IQ4_XS beats Qwen 3.6 27B despite the latter having MTP, on all of my actual coding task evaluations in real codebases. Qwen seems better at one-shotting things based on vague prompts to an acceptable degree, but thats literally not what I use thes…

Do you think the choice of quantization matters that much for other models? I've seen a lot of discussion about different quantization and FP formats but I feel totally unequipped to make an informed decision about what to try.

What's your evaluation setup like? It sounds like maybe the best thing to do is have a realistic evaluation that resembles your actual intended workload and workflow, and then just try everything.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#499

Post by yurishimo »

justaj wrote: post #466
davkan wrote: post #426

Earlier quoted context omitted.

There is currently no gpu in production that can max out the largest and fastest displays in graphically demanding games. We have monitors that are the equivalent of two 4k monitors side by side and run at 240hz. I have a 5080 and have to turn down settings to get 60fps in cyberpunk.

What if you do integer downscaling to 1080p on those 4k displays?

That's how most raytracing is done these days anyway. The game is rendered at a much lower resolution, the raytracing math is applied, and then it is upsampled to the target resolution.

If you set the target resolution to 1080p, not much changes in the render pipeline except the that final upscaling step. To get better quality, the lower resolution is bumped up so there is more data to work with for the upsampling, but the scaling performance can be very hit or miss depending on the game as the engine itself often can play a huge role in rendering performance.

As far as rendering the 1080p image at 4k, yea it works fine, but there will always be little artefacts that remain for those looking for them. 1440p seems to be the sweet spot for gamers today, but 4k is really nice for when you're not gaming as most online video is now made for dual use on televisions.

Post reply on HN