Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

news.ycombinator.com

Post reply on HN
Print view View original

311–320 of 620 posts

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#311

Post by ColonelPhantom »

system2 wrote: post #21

Until I can buy an 80GB VRAM GPU, I won't attempt to do it. A local LLM is always missing something that needs a bigger model.

Which model class requires an 80 GB VRAM GPU? From my perspective, popular models seem to be either in the ~30B range (Qwen3.6, Gemma 4), while the larger models (MiniMax, MiMo, StepFun, Deepseek) are in the multiple hundreds of billions parameters, for which 80 GB is simply too small.

You can just about reach the lower end of the latter category with a 128GB machine like a DGX Spark, Framework Desktop, or M5 Max, though those are usually not super fast. For the former category, you can easily run them fast with something like a 3090 or 5090, hell, probably even a 5060 Ti.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#312

Post by booi »

overgard wrote: post #207
rootlocus wrote: post #100

Earlier quoted context omitted.

2x RTX3090 are around $4400. Without any electricity costs or other parts, that's 3.6 years of $100/m claude.

Assuming the $100/m claude subscription is still around in three years.

we will be lucky if it's still around in 3 months..

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#314

Post by epolanski »

Not with a local one, but I moved to DeepSeek v4.

Albeit I plan to move to local ones when I will get my hands on a 256+ GB macbook.

Local inference is good enough to help me with my daily job, and doesn't turn me into an assistant to the LLM.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#315

Post by ojr »

Greenpants wrote: post #112

I have! I care about data privacy and LLMs being free. I'm using the Pi coding harness but containerized and sandboxed, to make sure it's running completely offline. On my Mac Studio with 128GB RAM (or MacBook with 36GB RAM) I'm using Qwen3.6 35b, with only 3b active parameters so that it runs really fast. I've done a complete redesign for my website's homepage and blog with Django + Wagtail. The latter is interestin…

I can use Gemini 3 Flash with the harness I built for around 8 years and still not exceed the cost of a Mac Studio with 128GB, the price for privacy is very high. Agentic flows that get stuck can be worked around but I prefer developer velocity.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#316

Post by carterschonwald »

lambda wrote: post #282
dnautics wrote: post #257

Earlier quoted context omitted.

> Qwen hybrid models don't handle prompt caching and instead re-process the context in full on every turn. I'm wondering if you were able to solve this and how? Isn't this the nature of how LLMs work? Or do you mean that it recalculates the entire KV cache instead of saving the old KV cache, in which case the problem is likely in your executor (llama.cpp, vllm, e.g.) configuration or capabilities?

So, one of the ways that this problem manifests is that most local models aren't trained on preserving the full reasoning between turns. Every turn, they skip passing the reasoning trace from previous turns to the the LLM. So if on one turn you have a long interleaved chain of reasoning and tool calls, then it responds to you, and then you give a new prompt to fix something, it has to re-process all of those tools ca…

thats a harness issue not a model issue. eg i have my own reasoninf harness that forced persisted cot

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#317

Post by epolanski »

jacobgold wrote: post #62
pierotofy wrote: post #36

Yes. Llama.cpp + Qwen3.6-35b (MTP) + OpenCode is quite capable and runs on a single RTX 3090 and is faster than most cloud models. Quality is like running edge models from 8-12 months ago. Setup details at https://github.com/pierotofy/LocalCodingLLM/

"Quality is like running edge models from 8-12 months ago." That sounds great for hobbyists but IMHO it wasn't until Opus 4.6 was released six months go (Dec 25, 2025) that we had a model good enough for professionals to use as a primary driver of their coding agents. That seems to be the threshold worth aiming for.

Why don't you people bother to try instead of chasing the latest shiny thing?

You must be the type of crowd that writes websites with React and Tailwind and pretend to be engineers and have an opinion on everything.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#318

Post by _bobm »

XCSme wrote: post #309
_bobm wrote: post #277

Earlier quoted context omitted.

> Can you give an example? Sure, connect opencode to an openai/chatgpt endpoint and use it. You will notice multiple "thinking" parts per "turn". I put all of these in quotation because... they are part of the orchestration game. For example, it is not known if the thinking parts of a particular turn are chain of thought thinking summaries or just plain response which is masquaraded and thus orchestrated into appeari…

> You will notice multiple "thinking" parts per "turn" I thought that was the code harness simply minifying the outputs. Many models now no longer return the entire chain-of-thought (to avoid distillation attacks). So yes, we don't get the raw LLM output, but I think it's just the thinking summarized, not a complex orchestration or different models. I do agree though that now cloud models are kind of a black box, tha…

idk what is "minifying outputs" in the context of what we are talking about. Opencode is opensource, you can find out what it is doing.

Last time I checked, OpenAI even send (in the response) the summary of the thinking part alreafy in markdown, so opencode has to remove the formatting to format it to their liking.

> Many models now no longer return the entire chain-of-thought (to avoid distillation attacks).

This is what they say: to avoid distillation attacks. And to some large extent this is true. I am saying there is a side- effect and this side- effect (depending on how tin-foilly you want to go) may be either a nice thing to have or it may be the "main reason" for all of this.

The side effect is splicing the inference, brokering requests, and what not, which brings huge benefits at scale.

This was my original point: openweights model to a sota model may be apples to oranges. So when will a local model catchup with its single cot run which is not even shaped properly: well never.

It is apples to oranges.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#319

Post by epolanski »

wsintra2022 wrote: post #297

Reading through these comments, I can't tell any more whats bots posting on behalf of the AI providers trying to dissuade or whether people just have had negative experiences with local ai models. IMO, Qwen 3.6 27B 8k quants running on a Mac Studio 64g ram, incredible?. No it is not frontier general super shit, its just good. That's it, its good. Its free and private and can take an experienced engineer from being la…

The software "engineering" field is filled with MIT Leetcode ninjas writing React+Tailwind memory leaking unusable slop, the bar is extremely low.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#320

Post by 3abiton »

I think nearly everyone mentioned Qwen, so my turn I guess. Qwen 3.6 35B Q8 (MTP), on a Strix Halo, with llama.cpp. Around 40-50 t/s. Really great pefromance, I get always suprised by its capability. I used with forge-code directly in zsh. For long context 150k+) it start degrading and forgetting.
Post reply on HN