[deleted]
Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
301–310 of 620 posts
Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#302- What would you say is the best model for coding at the moment that can run on a high end consumer GPU? (Assume an RTX 3090/4090 is available.)
- What "stack" do you recommend? Llama.cpp + OpenCode?
Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#303Earlier quoted context omitted.
2x RTX3090 is extremely overkill for gaming, you can run any released game on earth on ultra for much less
1x RTX3090 is absolutely not overkill for gaming however. Nowadays it's barely enough to get 60FPS in 4K in some recently released games. But the shocking part is that my 3090 is still probably worth as much as when I bought it about 4 years ago.
It's probably worth more now.
Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#304Earlier quoted context omitted.
> Qwen hybrid models don't handle prompt caching and instead re-process the context in full on every turn. I'm wondering if you were able to solve this and how? Isn't this the nature of how LLMs work? Or do you mean that it recalculates the entire KV cache instead of saving the old KV cache, in which case the problem is likely in your executor (llama.cpp, vllm, e.g.) configuration or capabilities?
So, one of the ways that this problem manifests is that most local models aren't trained on preserving the full reasoning between turns. Every turn, they skip passing the reasoning trace from previous turns to the the LLM. So if on one turn you have a long interleaved chain of reasoning and tool calls, then it responds to you, and then you give a new prompt to fix something, it has to re-process all of those tools ca…
wait do sota models use mamba-like SSMs? this is the first im hearing this
Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#305Earlier quoted context omitted.
I use Vulkan mostly instead of ROCm. Vulkan is actually a bit faster, paradoxically. I do switch out and try them both out, and it's not a huge difference, but I've been mostly saying on Vulkan. The re-processing context every turn problem is definitely something I've hit. Some of the causes have been solved upstream in llama.cpp; make sure you're up to date. But another cause of the issue that has a big effect is th…
+1 using llama.cpp Vulkan releases with the Qwen models - runs much better than the ROCm releases. I'll have to give the preserve_thinking a shot.
Thanks for sharing have been running ROCm primarily with Qwen 3.6 and Qwen Coder, on the runs much better statement is that a stability, performance or other capability your experiencing?
Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#306Is anyone using Aider?
Is there any decent CLI alternatives to it?
Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#307For personal use, yes. I replaced a $100/m subscription to claude in favor of running pi harness pointed at unsloth studio, using both qwen (unsloth/Qwen3.6-35B-A3B-MTP-GGUF) and gemma (unsloth/gemma-4-26B-A4B-it-GGUF) models, depending on my mood. I have a machine I built about 5 years ago with dual RTX3090s in it (I was going to build a new gaming machine anyways, and the llama release had just dropped so I tacked…
2x RTX3090 are around $4400. Without any electricity costs or other parts, that's 3.6 years of $100/m claude.
Those GPUs can also play video games or mine cryptocurrency. They can also be sold later.
We should own things, not rent them. We should all do what we can to keep the fabled 2030 agenda at bay.
Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#308I have not. We use openspec with our projects at work. To try and simulate a local rig without spending big cash. I use the hosted models and pay for them with the latest popular local model.
Most small local models don't get tool calling right, however the larger models are now doing this correctly now.
One thing local has not accounted for, is most productive engineers are running multiple cli chats at a time with git worktrees. I normally hover around 3 worktrees + cli-chats.
Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#309Earlier quoted context omitted.
> The SOTA models are a deep orchestration of multiple models operating together it isn't a single mode I don't understand, why does it make you think this is the case? > how can GPT send thinking parts one after another with a markdown header summary of the thinking block itself Can you give an example?
> Can you give an example? Sure, connect opencode to an openai/chatgpt endpoint and use it. You will notice multiple "thinking" parts per "turn". I put all of these in quotation because... they are part of the orchestration game. For example, it is not known if the thinking parts of a particular turn are chain of thought thinking summaries or just plain response which is masquaraded and thus orchestrated into appeari…
> You will notice multiple "thinking" parts per "turn"
I thought that was the code harness simply minifying the outputs. Many models now no longer return the entire chain-of-thought (to avoid distillation attacks). So yes, we don't get the raw LLM output, but I think it's just the thinking summarized, not a complex orchestration or different models.
I do agree though that now cloud models are kind of a black box, that's not only obfuscated but also changes over time. Companies seem to be changing model capabilities without notifying users, or even hiddenly serving completely different models. This is even worse via OpenRouter, with providers serving open-source models, some of them serve heavily quantized versions or even completely different models.
Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#310I have! I care about data privacy and LLMs being free. I'm using the Pi coding harness but containerized and sandboxed, to make sure it's running completely offline. On my Mac Studio with 128GB RAM (or MacBook with 36GB RAM) I'm using Qwen3.6 35b, with only 3b active parameters so that it runs really fast. I've done a complete redesign for my website's homepage and blog with Django + Wagtail. The latter is interestin…
I got a 48GB Ram MacBook, somehow I cannot even run a 20b model, I was suprised that you get 35b model locally.