Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code: connect to a local model when your quota runs out

boxc.net

Post reply on HN
Print view View original

151–160 of 223 posts

Re: Claude Code: connect to a local model when your quota runs out

#151

Post by PeterStuer »

andai wrote: post #86
paxys wrote: post #22

> Reduce your expectations about speed and performance! Wildly understating this part. Even the best local models (ones you run on beefy 128GB+ RAM machines) get nowhere close to the sheer intelligence of Claude/Gemini/Codex. At worst these models will move you backwards and just increase the amount of work Claude has to do when your limits reset.

Yeah this is why I ended up getting Claude subscription in the first place. I was using GLM on ZAI coding plan (jerry rigged Claude Code for $3/month), but finding myself asking Sonnet to rewrite 90% of the code GLM was giving me. At some point I was like "what the hell am I doing" and just switched. To clarify, the code I was getting before mostly worked, it was just a lot less pleasant to look at and work with. Mig…

My very first tests of local Qwen-coder-next yesterday found it quite capable of acceptably improving Python functions when given clear objectives.

I'm not looking for a vibe coding "one-shot" full project model. I'm not looking to replace GPT 5.2 or Opus 4.5. But having a local instance running some Ralph loop overnight on a specific aspect for the price of electricity is alluring.

Re: Claude Code: connect to a local model when your quota runs out

#152

Post by dust42 »

Gravey wrote: post #58
bityard wrote: post #52

Earlier quoted context omitted.

Correct, a rack full of datacenter equipment is not going to compete with anything that fits on your desk or lap. Well spotted. But as a counterpoint: there are whole communities of people in this space who get significant value from models they run locally. I am one of them.

Would you mind sharing your hardware setup and use case(s)?

The brand new Qwen3-Coder-Next runs at 300Tok/s PP and 40Tok/s on M1 64GB with 4-bit MLX quant. Together with Qwen Code (fork of Gemini) it is actually pretty capable.

Before that I used Qwen3-30B which is good enough for some quick javascript or Python, like 'add a new endpoint /api/foobar which does foobaz'. Also very decent for a quick summary of code.

It is 530Tok/s PP and 50Tok/s TG. If you have it spit out lots of the code that is just copy of the input, then it does 200Tok/s, i.e. 'add a new endpoint /api/foobar which does foobaz and return the whole file'

Re: Claude Code: connect to a local model when your quota runs out

#153

Post by sReinwald »

the_harpia_io wrote: post #136

[flagged]

Not saying the frontier models aren't smarter than the ones I can run on my two 4090s (they absolutely are) but I feel like you're exaggerating the security implications a bit.

We've seen some absolutely glaring security issues with vibe-coded apps / websites that did use Claude (most recently Moltbook).

No matter whether you're vibe coding with frontier models or local ones, you simply cannot rely on the model knowing what it is doing. Frankly, if you rely on the model's alignment training for writing secure authentication flows, you are doing it wrong. Claude Opus or Qwen3 Coder Next isn't responsible if you ship insecure code - you are.

Re: Claude Code: connect to a local model when your quota runs out

#154

Post by auggierose »

holoduke wrote: post #149
MuffinFlavored wrote: post #88

Earlier quoted context omitted.

Did you eventually move to a $20/mo Claude plan, $100/mo Claude plan, $200/mo, or API based? if API based, how much are you averaging a month?

I now have 3 x 100 plans. Only then I an able to full time use it. Otherwise I hit the limits. I am q heavy user. Often work on 5 apps at the same time.

Shouldn't the 200 plan give you 4x?? Why 3 x 100 then?

Re: Claude Code: connect to a local model when your quota runs out

#156

Post by heyyeah »

starkeeper wrote: post #45

Very cool. Anyone have guidance for using this with jetbrains IDE? It has a Claude Code plugin, but I think the setup is different for intelliJ... I know it has some configuration for local models, but the integrated Claude is such a superior experience then using their Junie, or just prompting diffs from the regular UI interface. HMMMM.... I guess I could try switching to the Claude Code CLI or other interface direc…

Thanks! (blog writer here)

Re: Claude Code: connect to a local model when your quota runs out

#157

Post by heyyeah »

bcyn wrote: post #67
eek2121 wrote: post #16

I gotta say, the local models are catching up quick. Claude is definitely still ahead, but things are moving right along.

Which models perform anywhere close to Opus 4.5? In my experience none of the local models are even in the same ballpark.

This week: look at Qwen3 Coder Next and GLM 4.7 but it's changing fast.

I wrote this for the scenario you've run out of quota for the day or week but want a back up plan to keep going to give some options with obvious speed and quality trade-offs. There is also always the option to upgrade if your project and use case needs Opus 4.5.

Re: Claude Code: connect to a local model when your quota runs out

#158

Post by heyyeah »

j45 wrote: post #98

Claude recently lets you top up with manual credits right in the web interface - it would be interesting if these were allowed to top up and unlock the max plans.

When you run out of quota it presents you with options to stop, upgrade -- or I added a third option in this blog to connect to a local model until your quota resets: https://boxc.net/blog/2026/claude-code-connecting-to-local-m...

Re: Claude Code: connect to a local model when your quota runs out

#159

Post by heyyeah »

swyx wrote: post #5

i mean the other obvious answer is to plug in to the other claude code proxies that other model companies have made for you: https://docs.z.ai/devpack/tool/claude https://www.cerebras.ai/blog/introducing-cerebras-code or i guess one of the hosted gpu providers if you're basically a homelabber and wanted an excuse to run quantized models on your own device go for it but dont lie and mutter under your own tin foil hat…

It's definitely a backup solution but even since I was drafting the blog, Qwen3 Coder Next was released. It's a functional stop gap if you want to keep things local. I try to be up front in the blog for people to "Reduce your expectations about speed and performance!"

(Also, I love your podcast!)

Re: Claude Code: connect to a local model when your quota runs out

#160

Post by amelius »

paxys wrote: post #22

> Reduce your expectations about speed and performance! Wildly understating this part. Even the best local models (ones you run on beefy 128GB+ RAM machines) get nowhere close to the sheer intelligence of Claude/Gemini/Codex. At worst these models will move you backwards and just increase the amount of work Claude has to do when your limits reset.

And at best?
Post reply on HN