Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Claude Code: connect to a local model when your quota runs out

boxc.net

Post reply on HN
Print view View original

201–210 of 223 posts

Re: Claude Code: connect to a local model when your quota runs out

#201

Post by VladVladikoff »

deaux wrote: post #122
Aurornis wrote: post #117

Earlier quoted context omitted.

> Kimi K2.5 is absolutely competitive for coding. Kimi K2.5 is good, but it's still behind the main models like Claude's offerings and GPT-5.2. Yes, I know what the benchmarks say, but the benchmarks for open weight models have been overpromising for a long time and Kimi K2.5 is no exception. Kimi K2.5 is also not something you can easily run locally without investing $5-10K or more. There are hosted options you can…

I disagree, based on having used it extensively over the last week. I find it to be at least as strong as Sonnet 4.5 and 5.2-Codex on the majority of tasks, often better. Note that even among the big 3, each of them has a domain where they're better than the other two. It's not better than Codex (x-)high at debugging non-UI code - but neither is Opus or Gemini. It's not better than Gemini at UI design - but neither i…

What hardware are you running it on?

Re: Claude Code: connect to a local model when your quota runs out

#202

Post by ProllyInfamous »

armcat wrote: post #200

Ollama also supports integration with Claude Code: https://docs.ollama.com/integrations/claude-code Essentially: `ollama launch claude`

I was really impressed with how Ollama 3 ran on an AMD VEGA64 (~2017 tech) with only 8gb of [HBM] RAM. It was definitely limited, but very local and helpful.

Re: Claude Code: connect to a local model when your quota runs out

#204

Post by anonymousDan »

the_harpia_io wrote: post #194
Zardoz84 wrote: post #137

Earlier quoted context omitted.

I would always prefer something local. By definition it's more secure, as you are not sending your code on the wire to a third party server, and hope that they comply with the "We will not train our models with your data".

[flagged]

I think you are deluded if you think the latter does not happen with hosted models.

Re: Claude Code: connect to a local model when your quota runs out

#207

Post by prmph »

AlwaysRock wrote: post #186

Not exactly the same but I wish copilot/github allowed you to have two plans. A company sponsored plan and your own plan. If I run out of requests on my company plan I should be able to use my own plan. Likewise, If I have 1 github account that is used for work and non work code, I should be able to route copilot to use a company or personal plan.

Maybe what you actually want is to simply be able to switch to a another account when credits on one run out.

Because mixing company and personal accounts might not be a good idea.

Re: Claude Code: connect to a local model when your quota runs out

#209

Post by cactusplant7374 »

paxys wrote: post #73
cactusplant7374 wrote: post #43

Earlier quoted context omitted.

Inference is profitable. Maybe we hit a limit and we don't need as many expensive training runs in the future.

Inference APIs are probably profitable, but I doubt the $20-$100 monthly plans are.

I wouldn’t be so sure. Most users aren’t going to use up their quota every week.

Re: Claude Code: connect to a local model when your quota runs out

#210

Post by mercutio2 »

throwa356262 wrote: post #143
mercutio2 wrote: post #53

Earlier quoted context omitted.

Yeah, the generosity of Anthropic is vastly less than OpenAI. Which is, itself, much less than Gemini (I've never paid Google a dime, I get hours of use out of gemini-cli every day). I run out of my weekly quota in 2-3 days, 5-hour quota in ~1 hour. And this is 1-2 tasks at a time, using Sonnet (Opus gets like 3 queries before I've used my quota). Right now OpenAI is giving away fairly generous free credits to get pe…

How recent is your information? Google significantly reduced the free quota and removed pro models from gemini cli some 2-3 moths ago. Also, Gemini models eat tokens like crazy. Something Codex and Code would do with 2K tokens takes Gemini 100K. Not sure why.

I guess I’ve never tried the pro models, because I’ve used gemini-cli free every day for the last three months or so.

It does eventually finish its quota, but then I just switch to a different Google account (which, amusingly, is what Gemini told me to do).

Happy to consume Google’s free tokens! The free model is a distant third for coding, but it’s fine for leaf node work in a larger project.

Post reply on HN