Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

news.ycombinator.com

Post reply on HN
Print view View original

151–160 of 620 posts

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#151

Post by jacobgold »

pierotofy wrote: post #127
jacobgold wrote: post #75

Earlier quoted context omitted.

That's cool if you prefer it, but it is hard to imagine it being a strictly rational choice when much better quality is available at a price that is small relative to the cost of an employee. Or is there something specific about your use-case?

To me, what's not rational is believing you must rent the tools of your trade while exposing all of your employer's intellectual property to a third party. Difference of opinion.

It's not my opinion that you "must" rent tools but it certainly is the pragmatic choice in 2026. I would be as happy as anyone for this situation to change and I expect it to at some point.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#152

Post by [deleted] »

papichulo4 wrote: post #72
kandros wrote: post #13

Earlier quoted context omitted.

I’d rather ask my butcher than Haiku for coding tasks

Agreed on this. Anthropic has now changed the verbiage on the definitions of the models under `/model` to say that Opus is for everyday usage, and Sonnet is for routine tasks. There's apparently a reason Sonnet and Haiku have been left in previous version #s. Still encouraging, though, that things are catching up. We can't expect $20k local setups to match $20bn compute clusters.

[deleted]

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#153

Post by coder543 »

mitchell_h wrote: post #22

Tried. The context windows just weren't big enough.

Qwen3.6-27B supports a 1 million token context window.

Of course, you have to have the right hardware to be able to run with a context window like that, as it takes about 100GB of memory on my DGX Spark to do that with full f16 KV cache on the q4_k_xl model.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#154

Post by bArray »

I'm in the middle of building my own based on LiquidAI/LFM2.5-1.2B-Instruct [1]. I run it on the CPU locally and get reasonable performance. I'm currently using it to solve small problems - but expanding it daily.

[1] https://huggingface.co/LiquidAI/LFM2.5-1.2B-Instruct

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#155

Post by lambda »

Greenpants wrote: post #112

I have! I care about data privacy and LLMs being free. I'm using the Pi coding harness but containerized and sandboxed, to make sure it's running completely offline. On my Mac Studio with 128GB RAM (or MacBook with 36GB RAM) I'm using Qwen3.6 35b, with only 3b active parameters so that it runs really fast. I've done a complete redesign for my website's homepage and blog with Django + Wagtail. The latter is interestin…

This is very similar to my setup. Pi in a container (I do let it have network access, just no access to creds or anything, only the one directory that I'm working on at the time and my ~/.pi directory), talking to llama.cpp in another container. I'm on a Strix Halo 128 GiB unified memory laptop.

I've never used the frontier models in earnest, I don't believe in using proprietary tools for my programming, so I can't really compare.

And I'm still a AI skeptic, so I'm doing more testing and kicking the tires than I am actually using it. That means I spend a lot of time trying to break various models, probe them for strengths and weaknesses, etc.

But I find that when I do try to use it for real for agentic coding, Qwen 3.6 35B-A3B is definitely the one I reach for the most often.

For other chat tasks and translation, I'll frequently use Gemma 4 31B.

For audio, I'll use Gemma 4 12B.

I keep a bunch of other models around to try out every once in a while (Qwen 3.5 122B-A10B, Qwen 3.6 27B, Nemotron 3 Super 122B-A12B, Step 3.7 Flash and Minimax M2.7 both at somewhat more aggressive quants, and GPT-OSS 120B if I want super fast but not terribly smart), but so far Qwen 3.6 35B-A3B is really the sweet spot for coding on a setup like this.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#156

Post by htrp »

acc_297 wrote: post #8

I've been wondering lately if it would help to take a medium sized model and either in cloud or some local setup actually do Reinforcement Learning from Human Feedback (RLHF) on every prompt as a chore - I don't know if trying to manually finetune a model to your use habits would ruin it or help - ideally if you were diligent you could get rid of some of the ticks that make models for the general public difficult to…

Cursor is doing that (i think with Fireworks as their provider)

https://cursor.com/blog/real-time-rl-for-composer

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#157

Post by coder543 »

blurbleblurble wrote: post #33

My experience is that it's not the models themselves that are limiting right now, it's the clunky alternative harnesses with weird missing features making for bad ergonomics around stuff like queue management, interruption, subagents, goals, etc.

I agree completely.

It's also annoying that OpenCode doesn't even try to support local LLMs properly.

Getting OpenCode to work is possible, but extremely manual and clunky to configure. I have written a script to automate converting my llama-server configs into an OpenCode config, and that helps, but it's not ideal.

I have seriously considered writing Yet Another Coding Harness in my free time. I have some ideas for what would make it nice.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#158

Post by nobody_r_knows »

Greenpants wrote: post #112

I have! I care about data privacy and LLMs being free. I'm using the Pi coding harness but containerized and sandboxed, to make sure it's running completely offline. On my Mac Studio with 128GB RAM (or MacBook with 36GB RAM) I'm using Qwen3.6 35b, with only 3b active parameters so that it runs really fast. I've done a complete redesign for my website's homepage and blog with Django + Wagtail. The latter is interestin…

[dead]

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#159

Post by jmuguy »

Greenpants wrote: post #112

I have! I care about data privacy and LLMs being free. I'm using the Pi coding harness but containerized and sandboxed, to make sure it's running completely offline. On my Mac Studio with 128GB RAM (or MacBook with 36GB RAM) I'm using Qwen3.6 35b, with only 3b active parameters so that it runs really fast. I've done a complete redesign for my website's homepage and blog with Django + Wagtail. The latter is interestin…

Given your knowledge on this - do you think we'll see an open source model with Opus levels of capability? IMO if/when this happens - I would 100% stop using Anthropic.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#160

Post by stymaar »

anana_ wrote: post #117
manmal wrote: post #106

Earlier quoted context omitted.

Have you tried the 27B dense version? It’s way better for coding.

Unfortunately on Strix Halo or any similar unified memory set up, dense models are gonna be dirt slow due to the tiny memory bandwidth... But I agree, 27B is superior.

Exactly. That's why I'm disappointed there wasn't a 122B version, it's 27B but for Strix Halo users.
Post reply on HN