Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A guide to local coding models

aiforswes.com

Post reply on HN
Print view View original

181–190 of 363 posts

Re: A guide to local coding models

#181

Post by kube-system »

amarant wrote: post #152

Buying a maxed out MacBook Pro seems like the most expensive way to go about getting the necessary compute. Apple is notorious for overcharging for hardware, especially on ram. I bet you could build a stationary tower for half the price with comparable hardware specs. And unless I'm missing something you should be able to run these things on Linux. Getting a maxed out non-apple laptop will also be cheaper for compara…

You need memory hooked up to the GPU. Apple’s unified memory is actually one of the cheaper ways to do this. On a typical x86-64 desktop, this means VRAM… for 100+ GB of VRAM you’re deep into tens of thousand of dollars.

Also, if you think Apple’s RAM prices are crazy… you might be surprised at what current DDR5 pricing is today. The $800 that Apple charges to upgrade a MBP from 64-128GB is the current price of 64GB desktop DDR5-6000. Which is actually slower memory than the 8533 MT/s memory you’re getting in the MacBook.

Re: A guide to local coding models

#183

Post by lodovic »

throwthrowuknow wrote: post #118
mudkipdev wrote: post #66

Earlier quoted context omitted.

Claude's $20 plan should be renamed to "trial". Try Opus and you will reach your limit in 10 minutes. With Sonnet, if you aren't clearing the context very often, you'll hit it within a few hours. I'm sympathetic to developers who are using this as their only AI subscription because while I was working on a challenging bug yesterday I reached the limit before it had even diagnosed the problem and had to switch to anot…

I half agree, but it should be called “Hobbiest” since that’s what it’s good for. 10 minutes is hyperbolic, I average 1h30m even when using plan mode first and front loading the context with dev diaries, git history, milestone documents and important excerpts from previous conversations. Something tells me your modules might be too big and need refactoring. That said, it’s a pain having to wait hours between sessions…

I actually get more mileage out of Claude using a Github Copilot subscription. The regular Claude Pro will give me an hour or up to 90 minutes max, before it reaches the cap. The Github version has a monthly limit for the Claude requests (100 "premium requests") which I find much easier to manage. I was about to switch to the max plan but this setup (both Claude pro and Github Copilot, costing 30 a month together) was just enough for my needs. With a bonus that I can try some of the other model offerings as well.

Re: A guide to local coding models

#184

Post by baconner »

alok-g wrote: post #174
brainless wrote: post #149

I do not spend $100/month. I spend for 1 Claude Pro subscription and then a (much cheaper) z.ai Coding Plan, which is like one fifth the cost. I use Claude for all my planning, create task documents and hand over to GLM 4.6. It has been my workhorse as a bootstrapped founder (building nocodo, think Lovable for AI agents).

I have heard about this approach elsewhere too. Could you please provide some more details on the set up steps and usage approach. I would like to replicate. Thanks.

There are a couple of decent approaches to having a planning/reviewer model set (eg. claude, codex, gemini) and an execution model (eg. glm 4.6, flash models, etc) workflow that I've tried. All three of these will let you live in a single coding cli but swap in different models for different tasks easily.

- claude code router - basically allows you to swap in other models using the real claude code cli and set up some triggers for when to use which one (eg. plan mode use real claude, non plan or with keywords use glm)

- opencode - this is what im mostly using now. similar to ccr but i find it a lot more reliable against alt models. thinking tasks go to claude, gemini, codex and lesser execution tasks go to glm 4.6 (on ceberas).

- sub-agent mcp - Another cool way is to use an mcp (or a skill or custom /command) that runs another agent cli for certain tasks. The mcp approach is neat because then your thinker agent like claude can decide when to call the execution agents, when to call in another smart model for a review of it's own thinking, etc instead of it being explicit choice from you. So you end up with the mcp + an AGENTS.md that instructs it to aggressively use the sub-agent mcp when it's a basic execution task, review, ...

I also find that with this setup just being able to tap in an alt model when one is stuck, or get review from an alt model can help keep things unstuck and moving.

Re: A guide to local coding models

#185

Post by mkesper »

r-w wrote: post #171
kristopolous wrote: post #166

Earlier quoted context omitted.

I'll get it up soon, probably should. This little snippet will help you though: $ man --html="$(which markitdown)" That goes man -> html -> markdown which is not only token efficient but also llms are pretty good at creating hierarchies from markdown

I bet you could do the same thing with pandoc and skip serializing to HTML entirely.

Apparently yes: https://pandoc.org/MANUAL.html#options

Re: A guide to local coding models

#186

Post by sheepscreek »

Aurornis wrote: post #139
sheepscreek wrote: post #131

Earlier quoted context omitted.

Not for Codex. Not even for Gemini/Antigravity! I am truly shocked by how much mileage I can get out of them. I recently bought the $200/mo OpenAI subscription but could barely use 10% of it. Now for over a month, I use codex for at least 2 hrs every day and have yet to reach the quota. With Gemini/Antigravity, there’s the added benefit of switching to Claude Code Opus 4.5 once you hit your Gemini quota, and Google i…

> I recently bought the $200/mo OpenAI subscription but could barely use 10% of it This entire comment is confusing. Why are you buying the $200/month plan if you’re only using 10% of it? I rotate providers. My comment above applies to all of them. It really depends on the work you’re doing and the codebase. There are tasks where I can get decent results and barely make the usage bar move. There are other tasks where…

I got it to try Atlas, their agentic browser, before it was open to Plus users. I convinced myself that I could use the additional capacity to multi-task and push through hard core problems without worrying about quota limits.

For context, this was a few months ago when GPT 5 was new and I was used to constantly hitting o3 limits. It was an experiment to see if the higher plan could pay for itself. It most certainly can but I realized that I just don’t need it. My workflow has evolved into switching between different agents on the same project. So now I have much less of a need for any one.

Re: A guide to local coding models

#187

Post by skhameneh »

thehamkercat wrote: post #16
ZeroCool2u wrote: post #10

Earlier quoted context omitted.

LMStudio is so much better than Ollama it's silly it's not more popular.

LMStudio is not open source though, ollama is but people should use llama.cpp instead

ik_llama is almost always faster when tuned. However, when untuned I've found them to be very similar in performance with varied results as to which will perform better.

But vLLM and Sglang tend to be faster than both of those.

Re: A guide to local coding models

#188

Post by kube-system »

kelvie wrote: post #170
jszymborski wrote: post #144

I just got a RTX 5090, so I thought I'd see what all the fuss was about these AI coding tools. I've previously copy pasted back and forth from Claude but never used the instruct models. So I fired up Cline with gpt-oss-120b, asked it to tell me what a specific function does, and proceeded to watch it run `cat README.md` over and over again. I'm sure it's better with other the Qwen Coder models, but it was a pretty fu…

gpt-oss-120b doesn't fit on a 5090 without offloading or crazy quants -- or did you mean you ran it via openrouter or something?

Sounds like a crazy quant. IME 2 bit quants are pretty dumb.

Re: A guide to local coding models

#189

Post by sheepscreek »

1over137 wrote: post #172
Aurornis wrote: post #159

Earlier quoted context omitted.

> but apparently they tried the $200/mo subscription, but after seeing they don't need it This is why it’s confusing, though. Why start with the highest plan as the starting point when it’s so easy to upgrade?

Because you’re rich?

Not rich. I pay in Canadian dollars :(

I’m just a simple dude trying to optimize his life.

Re: A guide to local coding models

#190

Post by __m »

aquafox wrote: post #175
kristopolous wrote: post #91

Earlier quoted context omitted.

I use local models + openrouter free ones. My monthly spend on ai models is I'm not cheap, just ahead of the curve. With the collapse in inference cost, everything will be this eventually I'll basically do $ man tool | or even $ cat source | Things I used to do intensively I now do lazily. I've even made a IEITYuan/Yuan-embedding-2.0-en database of my manpages with chroma and then I can just ask my local documentatio…

> I'll basically do $ man tool | or even $ cat source | Could you please elaborate on this? Do I get this right that you can set up your your command line so that you can pipe something to a command that sends this something together with a question to an LLM? Or did you just mean that metaphorically? Sorry if this is a stupid question.

i guess op means: $ man tool | ai

where ai could be a simple shell script combining the argument with stdin

Post reply on HN