Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

news.ycombinator.com

Post reply on HN
Print view View original

271–280 of 620 posts

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#271

Post by gunapologist99 »

codinhood wrote: post #26

I don't think you're going to get many "true" answers to this. The opportunity cost of not using the latest and best models is just too much right now. Every month I research this and come to the same conclusion: the time, effort, and cost required to get local models (and the coding tools around them) to perform even close to Claude Code with sonnet/opus just not worth it right now. If it was, it would be distributi…

Rather than Occam, consider Pareto?

If you truly believe that it WILL get there within the next couple of years, then you might as well start playing with it now (and, yes, you will be very surprised, especially for shorter/smaller projects or nicely modularized larger projects)

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#272

Post by amarshall »

htrp wrote: post #80
bluejay2387 wrote: post #68

About 90% of my coding is on Qwen 3.6 27b and Open Code with some custom skills and Semble. It is NOT as smart as CC or Codex but its enough to get most of my work done. I didn't set out to replace CC and Codex (I have an RTX 6000 so the TPS is faster than I care about, but the RTX 6000 was originally for other work). I only tried this just to see how close you could get to a frontier model for coding as an experimen…

why 27b vs 35b? Is MoE that much worse for coding?

Can take the geometric mean of total and active parameters of MoE to get approximate equivalent quality to dense model params. So sqrt(35*10)≈18.7.

The trade-off of MoE is that it is worse but faster for the same total size.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#273

Post by lambda »

tracker1 wrote: post #241
lambda wrote: post #196

Earlier quoted context omitted.

That should do pretty well. Memory bandwidth is the biggest bottleneck for token generation, at 644 GB/s you should be able to do pretty well on a 9070, while prompt proessing is more compute bound and Nvidia tends to have the edge there. 16 GiB won't fit you much, so you'd probably want at least 2x, and preferably 3x of those, and then you need a motherboard, power, etc. that can handle that.

You can get an R9700 with 32gb vram for ~$1200-1400 depending on where you live, which is probably a better option for AI use than 2x 9070(xt)

Yeah, definitely.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#274

Post by SubiculumCode »

kpw94 wrote: post #133
horsawlarway wrote: post #49

For personal use, yes. I replaced a $100/m subscription to claude in favor of running pi harness pointed at unsloth studio, using both qwen (unsloth/Qwen3.6-35B-A3B-MTP-GGUF) and gemma (unsloth/gemma-4-26B-A4B-it-GGUF) models, depending on my mood. I have a machine I built about 5 years ago with dual RTX3090s in it (I was going to build a new gaming machine anyways, and the llama release had just dropped so I tacked…

> gemma (unsloth/gemma-4-26B-A4B-it-GGUF) models Since you're running quantized (at UD-Q4_K_XL) , check out the "qat" models (unsloth/gemma-4-26B-A4B-it-qat-GGUF) ! - https://huggingface.co/unsloth/gemma-4-26B-A4B-it-qat-GGUF (With "Jun 9 Update: Added MTP support.") - https://blog.google/innovation-and-ai/technology/developers-...

How is the the QAT models at coding? I looked for opinions since the release and haven't found much.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#275

Post by zakisaad »

jborak wrote: post #176

I'm using 4x RTX 5070's and first-gen AMD threadripper (1950X) to run Qwen3.6 27B (MTP) Q6_K with llama.cpp and it works great as a daily driver with Pi. Around 50-60 toks/sec. I also connect a few other applications to it such as OpenWeb UI and recently set up Bifrost, an LLM gateway, to be the primary access point for the models I serve. I've tried other models such as Qwen3.6 35B A3B and I've found that 27B works…

This is interesting to me - why'd you go with the 5070 for your 4x build?

At first thought, they are quite skewed toward compute (vs VRAM), which is great for gamers but not so great for running LLMs.

(I run a 5070 in my desktop)

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#276

Post by bluerooibos »

hparadiz wrote: post #187
Greenpants wrote: post #112

I have! I care about data privacy and LLMs being free. I'm using the Pi coding harness but containerized and sandboxed, to make sure it's running completely offline. On my Mac Studio with 128GB RAM (or MacBook with 36GB RAM) I'm using Qwen3.6 35b, with only 3b active parameters so that it runs really fast. I've done a complete redesign for my website's homepage and blog with Django + Wagtail. The latter is interestin…

I am right there with you. Mind-boggling. It's a indistinguishable from magic technology!! I tried running some basic tasks through Qwen with Opencode on a 10 year old dual Xeon server for shits and giggles. I gave it a simple task like "use ffprobe first but convert this webm to mp4" and it was able to complete the task with zero network calls outside my network. On 10 year old hardware. It took about 3 minutes to c…

> 10 year old dual Xeon server...On 10 year old hardware.

Hold on, what are the specs of your rig? How much RAM?

I've been considering getting an old refurbished 2018 Mac Mini with 64Gb of DDR4 RAM but everything I've read suggests this will be way slower than my 16Gb M1 Pro Macbook.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#277

Post by _bobm »

XCSme wrote: post #266
_bobm wrote: post #261

But, guys, when you say Claude/ GPT models, do you stop to think what are these "models"? One day I thought about how can GPT send thinking parts one after another with a markdown header summary of the thinking block itself. Just think about it. As a matter of fact, think about these operations, api endpoints, observe their output. These so called SOTA models are not what meets the eye, and are not at all comparable…

> The SOTA models are a deep orchestration of multiple models operating together it isn't a single mode I don't understand, why does it make you think this is the case? > how can GPT send thinking parts one after another with a markdown header summary of the thinking block itself Can you give an example?

> Can you give an example?

Sure, connect opencode to an openai/chatgpt endpoint and use it. You will notice multiple "thinking" parts per "turn".

I put all of these in quotation because... they are part of the orchestration game. For example, it is not known if the thinking parts of a particular turn are chain of thought thinking summaries or just plain response which is masquaraded and thus orchestrated into appearing as thinking.

Further notice the cadence, word choice and sentence formation. Notice sentence construction. Notice "thinking part" construction and sequencing.

There is pretty heavy orchestration.

> I don't understand, why does it make you think this is the case?

Because not all tokens are equal. And if you waste expensive tokens on mundane tasks you will go out of business. This is the reason.

As I said, if you observe the output from these api endpoints you will notice it.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#278

Post by milchek »

I’ve tried in a 36GB MacBook Pro and haven’t had much success beyond very basic work. Issue for me was the context runs out quick even with smaller models and it’s slower. To get some half decent performance I’d imagine you want 128gb memory and are spending a lot more on hardware. At that point it becomes a question on whether you’d rather have frontier models at a subscription or sink that money into your own equipment. Of course, for those with privacy in mind your only option is forking out the cash for the higher end machines.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#279

Post by chungus »

Yup, although technically not replaced because I never used either of those products because I don't like sending my code to their black box. I have 2x24GB AMD gpu's, gotten from gamers on my local marketplace, one is connected with a 40cm riser cable. Running Qwen 27B and am very happy with its performance. Q8 with 135k context (arbitrary number, I could push it to 256). I like to use qwen 35B3A for mapping out entire code paths through our relatively complicated codebase/infra at work.

I think it's so good that I now scour the local marketplaces for good buys on 24GB cards that don't seem run through by miners and the likes, to build an even bigger rig for parallel execution.

Power usage is also totally not an issue, AI workload is very different from gaming.

tldr llama.cpp-vulkan with opencode on total 48GB VRAM AMD cards on arch btw.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#280

Post by jderekw »

Running AMD Lemonade as the daily rig, Started with Ollama then over to LMStudio and now standardized on AMD Lemonade which has been helpful to monitor cRAM, CPU, GPU and gRam. The multi-models on Lemonade make it straight forward to run a stack for LLM, Voice to Text, NPU, and Image Generation. Platform also works with Nvidia, Apple, Intel and AMD chip sets.
Post reply on HN