Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

news.ycombinator.com

Post reply on HN
Print view View original

331–340 of 620 posts

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#331

Post by jrm4 »

codinhood wrote: post #85
jrm4 wrote: post #45

Earlier quoted context omitted.

But you're pretty much measuring opportunity cost in tokens per second, no? I think it strongly remains to be seen whether e.g. tokens per second (multiplied or whatever by percieved quality of private model) actually means "better or more useful output." I strongly suspect it does not. (though I also strongly suspect this will be very difficult to measure because the incentive to lie about metrics here will be so st…

If you’re arguing that model metrics don’t necessarily translate into useful output, I agree. That’s not how I measure the success of a mode and not really the point I'm trying to make. I try to set things up and test it on my actual projects. What I’m saying is that if local models were actually comparable to Claude Code in practice, we wouldn’t be having threads like this. It would be obvious to the people using th…

Having, e.g. seen Microsoft maintain a monopoly for well over a decade, there's nothing in my experience that suggests that "quality always beats hype" is remotely true.

It's entirely possible Claude is just winning the hype game.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#332

Post by nl »

dnautics wrote: post #304
lambda wrote: post #282

Earlier quoted context omitted.

So, one of the ways that this problem manifests is that most local models aren't trained on preserving the full reasoning between turns. Every turn, they skip passing the reasoning trace from previous turns to the the LLM. So if on one turn you have a long interleaved chain of reasoning and tool calls, then it responds to you, and then you give a new prompt to fix something, it has to re-process all of those tools ca…

wait do sota models use mamba-like SSMs? this is the first im hearing this

Qwen 3.5 and above use Gated DeltaNet which alternate attention and SSM layers:

https://sebastianraschka.com/llms-from-scratch/ch04/08_delta...

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#334

Post by gr_norm »

orangeisthe wrote: post #237
xgulfie wrote: post #231

Earlier quoted context omitted.

I don't need a Ferrari to get to work

But you need the best tools to do the job

You need tools sufficient to do the job in an economical way, optimizing for both cost and quality. That is what 'best' means. We don't give every engineer all the resources under the sun, only what is appropriate.

I suspect many will realize millions more dollars are being spent than needed to achieve the highest marginal productivity gains, and reallocate accordingly. Who wants more of their money going to developer tooling, rather than bonuses?

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#335

Post by russelg »

mgsram wrote: post #284

I have been using local LLMs for about a year and I have settled now on Qwen3.6 27b dense model in GGUF on Mac Studio with 512G of RAM with open code as the harness and llmster(LM Studio). I have also used the Qwen 3.6 35B-A3B but the dense model's accuracy is next level with the tradeoff being tokens/sec. With the Qwen3.6 27b, I usually get anywhere from 25-40 tokens/second. Initially I used them for simple tools bu…

Since you have 512GB, might be worth looking into running deepseek4: https://github.com/antirez/ds4

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#336

Post by lowbloodsugar »

augusto-moura wrote: post #141
freetonik wrote: post #114

Earlier quoted context omitted.

That's also years of top tier PC gaming, if you're into that.

2x RTX3090 is extremely overkill for gaming, you can run any released game on earth on ultra for much less

I can’t run 4k HDR cyberpunk 2077 at 240hz with path tracing. I’m managing ~120fps. I’ve got a Blackwell 6000. I didn’t buy it for games, but there are still games and setups where the GPU is the bottleneck. I don’t even have an 8k TV.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#337

Post by _bobm »

XCSme wrote: post #324
_bobm wrote: post #318

Earlier quoted context omitted.

idk what is "minifying outputs" in the context of what we are talking about. Opencode is opensource, you can find out what it is doing. Last time I checked, OpenAI even send (in the response) the summary of the thinking part alreafy in markdown, so opencode has to remove the formatting to format it to their liking. > Many models now no longer return the entire chain-of-thought (to avoid distillation attacks). This is…

So, are you saying that local models are maybe better than we give them credit? Because with some extra orchestration/processing we could improve the results?

Yes, local models have already all that is needed, they have all the prerequisites.

But what they do not have is the correct shape, the correct approach. This is missing and it shows on multiple scales: it shows in the COT, it shows in the output itself, it shows in the infra to serve the models, it shows in the model orchestration.

This is what anthropic said one year ago:

> Finally, we've introduced thinking summaries for Claude 4 models that use a smaller model to condense lengthy thought processes. This summarization is only needed about 5% of the time—most thought processes are short enough to display in full. Users requiring raw chains of thought for advanced prompt engineering can contact sales about our new Developer Mode to retain full access.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#340

Post by henrixd »

I have been heavily relying on Qwen3.6-27B-UD-Q4_K_XL.gguf -model and Pi agent (https://pi.dev/) for local tasks and coding. I have used llama-cpp-turboquant fork with some custom cherrypicked MTP patches from another fork.

I'm running this on V100 32GB (~900GB/s memory bandwidth) with 200,000 context window, --spec-type mpt --spec-draft-n-max 3 --spec-draft-n-min 0 --cache-type-k turbo3 --cache-type-v turbo3 to mention most relevant parts.

I usually get somewhere 45-60 t/s. I believe that speed could be improved slightly by switching to ik_llama.cpp fork and Qwen3.6-27B-IQ4_NL.gguf -model but there's no turboquant support and it's with some other tradeoffs too.

Post reply on HN