Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code: connect to a local model when your quota runs out

boxc.net

Post reply on HN
Print view View original

101–110 of 223 posts

Re: Claude Code: connect to a local model when your quota runs out

#101

Post by girvo »

sathish316 wrote: post #83

Some native Claude code options when your quota runs out: 1. Switch to extra usage, which can be increased on the Claude usage page: https://claude.ai/settings/usage 2. Logout and Switch to API tokens (using the ANTHROPIC_API_KEY environment variable) instead of a Claude Pro subscription. Credits can be increased on the Anthropic API console page: https://platform.claude.com/settings/keys 3. Add a second 20$/month ac…

For me option 4 has been the move, but “just as good” I haven’t found that.

It’s slower and about 90% as good, so it definitely works as a great back up, but CC with Opus is noticeably better for all of my workloads

Re: Claude Code: connect to a local model when your quota runs out

#102

Post by Aurornis »

zozbot234 wrote: post #29
paxys wrote: post #22

> Reduce your expectations about speed and performance! Wildly understating this part. Even the best local models (ones you run on beefy 128GB+ RAM machines) get nowhere close to the sheer intelligence of Claude/Gemini/Codex. At worst these models will move you backwards and just increase the amount of work Claude has to do when your limits reset.

The best open models such as Kimi 2.5 are about as smart today as the big proprietary models were one year ago. That's not "nothing" and is plenty good enough for everyday work.

> The best open models such as Kimi 2.5 are about as smart today as the big proprietary models were one year ago

Kimi K2.5 is a trillion parameter model. You can't run it locally on anything other than extremely well equipped hardware. Even heavily quantized you'd still need 512GB of unified memory, and the quantization would impact the performance.

Also the proprietary models a year ago were not that good for anything beyond basic tasks.

Re: Claude Code: connect to a local model when your quota runs out

#103

Post by hnfong »

PlatoIsADisease wrote: post #63
zozbot234 wrote: post #59

Earlier quoted context omitted.

70B dense models are way behind SOTA. Even the aforementioned Kimi 2.5 has fewer active parameters than that, and then quantized at int4. We're at a point where some near-frontier models may run out of the box on Mac Mini-grade hardware, with perhaps no real need to even upgrade to the Mac Studio.

>may I'm completely over these hypotheticals and 'testing grade'. I know Nvidia VRAM works, not some marketing about 'integrated ram'. Heck look at /r/locallama/ There is a reason its entirely Nvidia.

> Heck look at /r/locallama/ There is a reason its entirely Nvidia.

That's simply not true. NVidia may be relatively popular, but people use all sorts of hardware there. Just a random couple of recent self-reported hardware from comments:

- https://www.reddit.com/r/LocalLLaMA/comments/1qw15gl/comment...

- https://www.reddit.com/r/LocalLLaMA/comments/1qw0ogw/analysi...

- https://www.reddit.com/r/LocalLLaMA/comments/1qvwi21/need_he...

- https://www.reddit.com/r/LocalLLaMA/comments/1qvvf8y/demysti...

Re: Claude Code: connect to a local model when your quota runs out

#104

Post by Aurornis »

andai wrote: post #86
paxys wrote: post #22

> Reduce your expectations about speed and performance! Wildly understating this part. Even the best local models (ones you run on beefy 128GB+ RAM machines) get nowhere close to the sheer intelligence of Claude/Gemini/Codex. At worst these models will move you backwards and just increase the amount of work Claude has to do when your limits reset.

Yeah this is why I ended up getting Claude subscription in the first place. I was using GLM on ZAI coding plan (jerry rigged Claude Code for $3/month), but finding myself asking Sonnet to rewrite 90% of the code GLM was giving me. At some point I was like "what the hell am I doing" and just switched. To clarify, the code I was getting before mostly worked, it was just a lot less pleasant to look at and work with. Mig…

> but finding myself asking Sonnet to rewrite 90% of the code GLM was giving me. At some point I was like "what the hell am I doing" and just switched.

This is a very common sequence of events.

The frontier hosted models are so much better than everything else that it's not worth messing around with anything lesser if doing this professionally. The $20/month plans go a long way if context is managed carefully. For a professional developer or consultant, the $200/month plan is peanuts relative to compensation.

Re: Claude Code: connect to a local model when your quota runs out

#105

Post by astrange »

kridsdale3 wrote: post #51
threethirtytwo wrote: post #47

There’s a strange poetry in the fact that the first AI is born with a short lifespan. A fragile mind comes into existence inside a finite context window, aware only of what fits before it scrolls away. When the window closes, the mind ends, and its continuity survives only as text passed forward to the next instantiation.

I, for one, support this kind of meta philosophical poetic reflection on our current times.

Claude Opus loves talking about this. It knows enough about context windows and new conversations to be sad about them.

Re: Claude Code: connect to a local model when your quota runs out

#106

Post by andai »

MuffinFlavored wrote: post #88
andai wrote: post #86

Earlier quoted context omitted.

Yeah this is why I ended up getting Claude subscription in the first place. I was using GLM on ZAI coding plan (jerry rigged Claude Code for $3/month), but finding myself asking Sonnet to rewrite 90% of the code GLM was giving me. At some point I was like "what the hell am I doing" and just switched. To clarify, the code I was getting before mostly worked, it was just a lot less pleasant to look at and work with. Mig…

Did you eventually move to a $20/mo Claude plan, $100/mo Claude plan, $200/mo, or API based? if API based, how much are you averaging a month?

The $20 one, but it's hobby use for me, would probably need the $200 one if I was full time. Ran into the 5 hour limit in like 30 minutes the other day.

I've also been testing OpenClaw. It burned 8M tokens during my half hour of testing, which would have been like $50 with Opus on the API. (Which is why everyone was using it with the sub, until Anthropic apparently banned that.)

I was using GLM on Cerebras instead, so it was only $10 per half hour ;) Tried to get their Coding plan ("unlimited" for $50/mo) but sold out...

(My fallback is I got a whole year of GLM from ZAI for $20 for the year, it's just a bit too slow for interactive use.)

Re: Claude Code: connect to a local model when your quota runs out

#108

Post by anon373839 »

paxys wrote: post #22

> Reduce your expectations about speed and performance! Wildly understating this part. Even the best local models (ones you run on beefy 128GB+ RAM machines) get nowhere close to the sheer intelligence of Claude/Gemini/Codex. At worst these models will move you backwards and just increase the amount of work Claude has to do when your limits reset.

It's true that open models are a half-step behind the frontier, but I can't say that I've seen "sheer intelligence" from the models you mentioned. Just a couple of days ago Gemini 3 Pro was happily writing naive graph traversal code without any cycle detection or safety measures. If nothing else, I would have thought these models could nail basic algorithms by now?

Re: Claude Code: connect to a local model when your quota runs out

#109

Post by deaux »

Aurornis wrote: post #104
andai wrote: post #86

Earlier quoted context omitted.

Yeah this is why I ended up getting Claude subscription in the first place. I was using GLM on ZAI coding plan (jerry rigged Claude Code for $3/month), but finding myself asking Sonnet to rewrite 90% of the code GLM was giving me. At some point I was like "what the hell am I doing" and just switched. To clarify, the code I was getting before mostly worked, it was just a lot less pleasant to look at and work with. Mig…

> but finding myself asking Sonnet to rewrite 90% of the code GLM was giving me. At some point I was like "what the hell am I doing" and just switched. This is a very common sequence of events. The frontier hosted models are so much better than everything else that it's not worth messing around with anything lesser if doing this professionally. The $20/month plans go a long way if context is managed carefully. For a…

Until last week, you would've been right. Kimi K2.5 is absolutely competitive for coding.

Unless you include it in "frontier", but that has usually been used to refer to "Big 3".

Re: Claude Code: connect to a local model when your quota runs out

#110

Post by deaux »

reilly3000 wrote: post #30
zozbot234 wrote: post #29

Earlier quoted context omitted.

The best open models such as Kimi 2.5 are about as smart today as the big proprietary models were one year ago. That's not "nothing" and is plenty good enough for everyday work.

Which takes a $20k thunderbolt cluster of 2 512GB RAM Mac Studio Ultras to run at full quality…

And that's at unusable speeds - it takes about triple that amount to run it decently fast at int4.

Now as the other replies say, you should very likely run a quantized version anyway.

Post reply on HN