Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

news.ycombinator.com

Post reply on HN
Print view View original

221–230 of 620 posts

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#221

Post by Greenpants »

yieldcrv wrote: post #216
Greenpants wrote: post #112

I have! I care about data privacy and LLMs being free. I'm using the Pi coding harness but containerized and sandboxed, to make sure it's running completely offline. On my Mac Studio with 128GB RAM (or MacBook with 36GB RAM) I'm using Qwen3.6 35b, with only 3b active parameters so that it runs really fast. I've done a complete redesign for my website's homepage and blog with Django + Wagtail. The latter is interestin…

> It gets into loops quite often matches my experience and a deal breaker also the context window sizes are too low. I can't operate in 65,000 windows any more because even just reading the code's file structure overruns it and gets me nowhere. Definitely its own art form. 200k context windows and above for me now I saw a paper last night that should help this a lot though

I get that it's a deal breaker to some; it definitely requires patience.

In Pi, /new is my best friend and most-used command for sure. For simple tasks (I decompose complex ones anyway since I don't trust small local LLMs to do this for me), the model doesn't need much context, given that I'm proficient in my codebase myself: "I'd like Feature X. Look into files 1, 2 and 3 to make your edits."

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#224

Post by nyxtom »

Greenpants wrote: post #112

I have! I care about data privacy and LLMs being free. I'm using the Pi coding harness but containerized and sandboxed, to make sure it's running completely offline. On my Mac Studio with 128GB RAM (or MacBook with 36GB RAM) I'm using Qwen3.6 35b, with only 3b active parameters so that it runs really fast. I've done a complete redesign for my website's homepage and blog with Django + Wagtail. The latter is interestin…

Have you found that being much more spec driven helps guide it better?

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#225

Post by 627467 »

So, everyone has different context, but how free is free running these local models? Like having a power hungry machine always on in the cupboard?

How much does this ware out the hardware?

Also, if privacy is the main reason for running local models, why not use venice.ai and equivalent?

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#226

Post by kennywinker »

yieldcrv wrote: post #216
Greenpants wrote: post #112

I have! I care about data privacy and LLMs being free. I'm using the Pi coding harness but containerized and sandboxed, to make sure it's running completely offline. On my Mac Studio with 128GB RAM (or MacBook with 36GB RAM) I'm using Qwen3.6 35b, with only 3b active parameters so that it runs really fast. I've done a complete redesign for my website's homepage and blog with Django + Wagtail. The latter is interestin…

> It gets into loops quite often matches my experience and a deal breaker also the context window sizes are too low. I can't operate in 65,000 windows any more because even just reading the code's file structure overruns it and gets me nowhere. Definitely its own art form. 200k context windows and above for me now I saw a paper last night that should help this a lot though

Qwen3.6-35b handles 256k context fine if you’ve got room for it. I’m running it with 128k context with just 16gb vram.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#227

Post by snake_n_my_boot »

daveidol wrote: post #79
pierotofy wrote: post #36

Yes. Llama.cpp + Qwen3.6-35b (MTP) + OpenCode is quite capable and runs on a single RTX 3090 and is faster than most cloud models. Quality is like running edge models from 8-12 months ago. Setup details at https://github.com/pierotofy/LocalCodingLLM/

Do you do your dev work on the windows machine (referenced in the docs), or do you remotely access it from a separate machine? I ask because I have a RTX 3090 kicking around in a gaming desktop, but I don't use it for any dev work (I use a Macbook Pro).

I have a similar set up and have been using it to learn and tinker with open models. I run Ollama on the gaming desktop and point OpenCode to it from my MacBook. Works nicely for me so far.

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#228

Post by AH4oFVbPT4f8 »

xeonax wrote: post #93
AH4oFVbPT4f8 wrote: post #78

Ollama + Hermes on M5 Max 128GB using .NET using Qwen 3.6:35b-a3b as the primary model to do the work. I might use 27b to plan what to do.

Whats .NET doing in between?

Sorry, I meant to say I was writing .NET C# with the setup

Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?

#230

Post by monirmamoun »

akulbe wrote: post #150
pierotofy wrote: post #61

Earlier quoted context omitted.

Yep. Screw Anthropic, CloseAI and all other rent seekers in this space.

I have an M2 Max MBP with 96GB of RAM. What models and setup would you use for this kind of configuration?

download LM Studio to play with, and it will let you search for models... try Qwen3.6-35B-A3B at 4,5 or 6 bits (6 bit XL is near perfect) and use pi coder or another harness to access it... you can also try Unsloth studio and try same model to start. LM Studio slighter easier to use, Unsloth probably better quality. Neither one is super great quality by the way (meaning: they crash or act weirdly too often to be full production solutions, but can work for local coding). ONCE YOU DOWNLOAD EITHER APP... it will let you search huggingface for the models. Just type qwen to start looking and ... start messing around. And you connect the pi coder harness using the http interface that LM Studio and Unsloth offer to the engine API, so make sure you figure out that url and turn it on... something like 127.0.0.1:1234/api would be a typical IP (localhost) and port (1234 is used by LM Studio)
Post reply on HN