Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code: connect to a local model when your quota runs out

boxc.net

Post reply on HN
Print view View original

141–150 of 223 posts

Re: Claude Code: connect to a local model when your quota runs out

#141

Post by EagnaIonat »

paxys wrote: post #22

> Reduce your expectations about speed and performance! Wildly understating this part. Even the best local models (ones you run on beefy 128GB+ RAM machines) get nowhere close to the sheer intelligence of Claude/Gemini/Codex. At worst these models will move you backwards and just increase the amount of work Claude has to do when your limits reset.

The secret is to not run out of quota.

Instead have Claude know when to offload work to local models and what model is best suited for the job. It will shape the prompt for the model. Then have Claude review the results. Massive reduction in costs.

btw, at least on Macbooks you can run good models with just M1 32GB of memory.

Re: Claude Code: connect to a local model when your quota runs out

#142

Post by davidwritesbugs »

andai wrote: post #86
paxys wrote: post #22

> Reduce your expectations about speed and performance! Wildly understating this part. Even the best local models (ones you run on beefy 128GB+ RAM machines) get nowhere close to the sheer intelligence of Claude/Gemini/Codex. At worst these models will move you backwards and just increase the amount of work Claude has to do when your limits reset.

Yeah this is why I ended up getting Claude subscription in the first place. I was using GLM on ZAI coding plan (jerry rigged Claude Code for $3/month), but finding myself asking Sonnet to rewrite 90% of the code GLM was giving me. At some point I was like "what the hell am I doing" and just switched. To clarify, the code I was getting before mostly worked, it was just a lot less pleasant to look at and work with. Mig…

Similar experience to me. I tend to let glm-4.7 have a go at the problem then if it keeps having to try I'll switch to Sonnet or Opus to solve it. Glm is good for the low hanging fruit and planning

Re: Claude Code: connect to a local model when your quota runs out

#143

Post by throwa356262 »

mercutio2 wrote: post #53
raw_anon_1111 wrote: post #9

Or just don’t use Claude Code and use Codex CLI. I have yet to hit a quota with Codex working all day. I hit the Claude limits within an hour or less. This is with my regular $20/month ChatGpT subscription and my $200 a year (company reimbursed) Claude subscription.

Yeah, the generosity of Anthropic is vastly less than OpenAI. Which is, itself, much less than Gemini (I've never paid Google a dime, I get hours of use out of gemini-cli every day). I run out of my weekly quota in 2-3 days, 5-hour quota in ~1 hour. And this is 1-2 tasks at a time, using Sonnet (Opus gets like 3 queries before I've used my quota). Right now OpenAI is giving away fairly generous free credits to get pe…

How recent is your information?

Google significantly reduced the free quota and removed pro models from gemini cli some 2-3 moths ago.

Also, Gemini models eat tokens like crazy. Something Codex and Code would do with 2K tokens takes Gemini 100K. Not sure why.

Re: Claude Code: connect to a local model when your quota runs out

#144

Post by ianlevesque »

deaux wrote: post #122
Aurornis wrote: post #117

Earlier quoted context omitted.

> Kimi K2.5 is absolutely competitive for coding. Kimi K2.5 is good, but it's still behind the main models like Claude's offerings and GPT-5.2. Yes, I know what the benchmarks say, but the benchmarks for open weight models have been overpromising for a long time and Kimi K2.5 is no exception. Kimi K2.5 is also not something you can easily run locally without investing $5-10K or more. There are hosted options you can…

I disagree, based on having used it extensively over the last week. I find it to be at least as strong as Sonnet 4.5 and 5.2-Codex on the majority of tasks, often better. Note that even among the big 3, each of them has a domain where they're better than the other two. It's not better than Codex (x-)high at debugging non-UI code - but neither is Opus or Gemini. It's not better than Gemini at UI design - but neither i…

Yeah Kimi-K2.5 is the first open weights model that actually feels competitive with the closed models, and I've tried a lot of them now.

Re: Claude Code: connect to a local model when your quota runs out

#145

Post by cracki »

anon373839 wrote: post #108
paxys wrote: post #22

> Reduce your expectations about speed and performance! Wildly understating this part. Even the best local models (ones you run on beefy 128GB+ RAM machines) get nowhere close to the sheer intelligence of Claude/Gemini/Codex. At worst these models will move you backwards and just increase the amount of work Claude has to do when your limits reset.

It's true that open models are a half-step behind the frontier, but I can't say that I've seen "sheer intelligence" from the models you mentioned. Just a couple of days ago Gemini 3 Pro was happily writing naive graph traversal code without any cycle detection or safety measures. If nothing else, I would have thought these models could nail basic algorithms by now?

Did it have reason to assume the graph to be a certain type, such as directed or acyclic?

Re: Claude Code: connect to a local model when your quota runs out

#146

Post by triage8004 »

Aurornis wrote: post #117
deaux wrote: post #109

Earlier quoted context omitted.

Until last week, you would've been right. Kimi K2.5 is absolutely competitive for coding. Unless you include it in "frontier", but that has usually been used to refer to "Big 3".

> Kimi K2.5 is absolutely competitive for coding. Kimi K2.5 is good, but it's still behind the main models like Claude's offerings and GPT-5.2. Yes, I know what the benchmarks say, but the benchmarks for open weight models have been overpromising for a long time and Kimi K2.5 is no exception. Kimi K2.5 is also not something you can easily run locally without investing $5-10K or more. There are hosted options you can…

Disagree it's behind gpt top models. It's just slightly behind opus

Re: Claude Code: connect to a local model when your quota runs out

#147

Post by BuildTheRobots »

EagnaIonat wrote: post #141
paxys wrote: post #22

> Reduce your expectations about speed and performance! Wildly understating this part. Even the best local models (ones you run on beefy 128GB+ RAM machines) get nowhere close to the sheer intelligence of Claude/Gemini/Codex. At worst these models will move you backwards and just increase the amount of work Claude has to do when your limits reset.

The secret is to not run out of quota. Instead have Claude know when to offload work to local models and what model is best suited for the job. It will shape the prompt for the model. Then have Claude review the results. Massive reduction in costs. btw, at least on Macbooks you can run good models with just M1 32GB of memory.

I don't suppose you could point to any resources on where I could get started. I have a M2 with 64gb of unified memory and it'd be nice to make it work rather than burning Github credits.

Re: Claude Code: connect to a local model when your quota runs out

#148

Post by ateevchopra »

One workaround that’s worked well for me is maintaining two Claude Code subscriptions instead of relying on just one.

When I hit the usage limit on the first account, I simply switch to the second and continue working. Since Claude stores progress locally rather than tying it to a specific account, the session picks up right where it left off. That makes it surprisingly seamless to keep momentum without waiting for limits to reset.

Re: Claude Code: connect to a local model when your quota runs out

#149

Post by holoduke »

MuffinFlavored wrote: post #88
andai wrote: post #86

Earlier quoted context omitted.

Yeah this is why I ended up getting Claude subscription in the first place. I was using GLM on ZAI coding plan (jerry rigged Claude Code for $3/month), but finding myself asking Sonnet to rewrite 90% of the code GLM was giving me. At some point I was like "what the hell am I doing" and just switched. To clarify, the code I was getting before mostly worked, it was just a lot less pleasant to look at and work with. Mig…

Did you eventually move to a $20/mo Claude plan, $100/mo Claude plan, $200/mo, or API based? if API based, how much are you averaging a month?

I now have 3 x 100 plans. Only then I an able to full time use it. Otherwise I hit the limits. I am q heavy user. Often work on 5 apps at the same time.

Re: Claude Code: connect to a local model when your quota runs out

#150

Post by EagnaIonat »

BuildTheRobots wrote: post #147
EagnaIonat wrote: post #141

Earlier quoted context omitted.

The secret is to not run out of quota. Instead have Claude know when to offload work to local models and what model is best suited for the job. It will shape the prompt for the model. Then have Claude review the results. Massive reduction in costs. btw, at least on Macbooks you can run good models with just M1 32GB of memory.

I don't suppose you could point to any resources on where I could get started. I have a M2 with 64gb of unified memory and it'd be nice to make it work rather than burning Github credits.

https://ollama.com

Although I'm starting to like LMStudio more, as it has more features that Ollama is missing.

https://lmstudio.ai

You can then get Claude to create the MCP server to talk to either. Then a CLAUDE.md that tells it to read the models you have downloaded, determine their use and when to offload. Claude will make all that for you as well.

Post reply on HN