[dead]
Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
601–610 of 620 posts
Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#602I have! I care about data privacy and LLMs being free. I'm using the Pi coding harness but containerized and sandboxed, to make sure it's running completely offline. On my Mac Studio with 128GB RAM (or MacBook with 36GB RAM) I'm using Qwen3.6 35b, with only 3b active parameters so that it runs really fast. I've done a complete redesign for my website's homepage and blog with Django + Wagtail. The latter is interestin…
How are you sandboxing your Pi coding harness? Directly only mounting certain folders, using capabilities to kill the network and not giving it all your shell env vars, that sort of thing? Or do you use a tool?
I'm using firejail to sandbox Opencode, for security and to keep the agents from personal data. I documented it in my blog [1].
[1] https://blog.fidelramos.net/software/how-i-sandbox-ai-agents...
Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#603Yes. I use Owen on my MacBook m1 (16gb) daily, running inside Ollama. Works well. Is not particularly fast, and I need to create a custom imagem that sets the temperature of the model to zero starting, so I don't get over creative with its bullshit, but it works reasonable week.
Secretly the problems many people have with agentic coding are related to poor choice of sampling settings, but the world will wait several more years before this is understood well. top_p and top_k are garbage but they are intentionally kept on purpose because subsequent methods enable coherent high temperature sampling, which is an absolute no go for alignment/safety reasons. The secret to actually good agentic out…
interesting. using Ollama I created a simple markdown file similar do a Dockerfile creating an agent called "boring as fuck".
This is the content
FROM qwen2.5-coder:7b
PARAMETER temperature 0
SYSTEM "You are a senior software engineer focused in php and Node.js.Your responses should be strictily technicals, without poetry or prose, and focused in safety. IF you are working with legacy code, just apply the changes with the best syntax possible."
Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#604[dead]
Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#605Earlier quoted context omitted.
2x RTX3090 are around $4400. Without any electricity costs or other parts, that's 3.6 years of $100/m claude.
My 3090 was $700 shipped. Shop around. I'd expect to pay "thousands" for the nodded 3090 with 48+ gigs of vram.
It was ~8mo ago.
Not anymore.
Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#606Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#607Cool as a fun experiment, but found it was not worth the hassle for me for coding. My machine is capable of running small models(<10B Parameters). I guess my experience would be different if I had more vram
Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#608I have! I care about data privacy and LLMs being free. I'm using the Pi coding harness but containerized and sandboxed, to make sure it's running completely offline. On my Mac Studio with 128GB RAM (or MacBook with 36GB RAM) I'm using Qwen3.6 35b, with only 3b active parameters so that it runs really fast. I've done a complete redesign for my website's homepage and blog with Django + Wagtail. The latter is interestin…
@greenpants, "Pi coding harness but containerized and sandboxed" care to address some specifics and/or reference implementation for this. may be a GITHub URL?
Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#609Earlier quoted context omitted.
That's not my experience, and the trajectory is good anyway - what doesn't work perfectly today will be just fine in a few months. In a quickly moving field, it's amazing how much money one can save by overcoming FOMO and not living on the bleeding edge. It's like waiting for Steam sales, the games will be just as good.
Curious what model you're using that works well on a 16GB card? I very much want to use my 5080 for inference, but everything I've tried so far has either just not been good enough or painfully slow.
gemma 4 12b
Re: Ask HN: Has anyone replaced Claude/GPT with a local model for daily coding?
#610Earlier quoted context omitted.
thats a harness issue not a model issue. eg i have my own reasoninf harness that forced persisted cot
Not a harness issue. The harness (pi in my case) passes back the cot for all previous turns. The jinja template is what renders the openai-format request sent by the harness, into the actual string of text that will be tokenized and fed to the model. For models without preserve thinking support, the jinja template drops the reasoning from all but the current turn. Here is the default jinja for Gemma 4: https://huggin…
my heavily modified test bed for of oh my pi fixes this