No, I mean, a one-click (or no-click) upgrade path. Having to hit a wall and then go get your API key and stuff sounds like a pain. It should just ask you if you want to switch to usage-based pricing when you hit the limit.
"no click" is insane, people pay $20/$100/$200 a month to have predictable bills, silently charging them unlimited money when you run out of quota is ridiculous. as to upgrade path, when you use up your quota, it tells you you can type /upgrade to switch to per-token API key billing.
Usage-based pricing is how most cloud providers work. It’s not ridiculous.
The calculation of hours is a little tough to imagine sometimes, is it the inference time itself, or the period of time used? Is there an average token cost per hour of use (average or explicit?)
It had better be inference time; I regularly have Claude call out to tools that take hours to run.
Yeah, I hope so too, but the silence might be a "it depends"
As a $20 month user, I can tell you in my experience it's "refactoring" jobs that really smash through those tokens quickly. If you do a "write a component that does this" kinda thing, you can use the $20 plan almost an unlimited amount of time. If you are doing "find all instances of ComponentFoo, change to ComponentBar, refactor each screen for correct usage of ComponentBar" kinda things, it's going to grep through…
Are people really using LLMs this way? Can't your IDE at least attempt global refactorings? I get that it's harder with dynamic languages like Python but using an LLM to rename a class seems like using a nuke to crack a nut.
For sure, it was a bit of a simplified example, but you can imagine prompts that are asking the LLM to touch dozens of files at once. A lot of people will use fairly high level requests, "make a screen in my app that does this" and that ends up making screens, components, queries, local state changes, etc etc. It adds up quick having so much in the context at once (and if you /clear the context and ask it to do anything more than one off changes in a file, it's gonna grep and glob it's way through your code looking for the right files)
Competition is the thing. Prices will drop as more AI code assistants get more adoption. Prices will probably also drop if anyone ever works out how to feasibly compete with NVIDIA. Not an expert here, but I expect they're worried about competition regulators, who will be watching them very closely.
> Prices will drop as more AI code assistants get more adoption. What's the reasoning behind this? They are already doing the efficient "economies of scale" thing and they are already at full capacity (hence rate limiting). The only way forward for this AI providers is to raise prices, not lower them.
The more AI assistants there are which are roughly equally competent, the more price becomes a factor. Mobility between providers is quick, it only takes one company willing to burn a lot of cash to win users or strategically hobble a competitor to start a price war. Maybe I'm wrong, but intuitively it feels like this will be the probable endgame.
You should consider self-hosting in the cloud. When you start coding run a script that spins up a new VM and loads the LLM of your choice, then run another script to spin it back down when you're done. For intermittent use this works great and is much cheaper than buying your own hardware, plus it's future proof. It does admittedly lack the cool factor of truly running locally though.
Too expensive from what I have seen - price for reasonably large GPU rigs that can host medium to large models is anywhere between ~5$/hr to ~9$/hr. That's ~40$-72$ for 8-hour working day or ~800$-1500$ for ~20 working days in a month. That's ~1000$ a month in average. This doesn't math for me.
I was assuming hobbyist use, so on the order of 2-4 hours every couple days
I'm probably not going to hit the weekly limit, but it makes me nervous that the limit is weekly as opposed to every 36 hours or something. If I do hit the limit, that's it for the entire week —a long time to be without a tool I've grown accustomed to! I feel like someone is going to reply that I'm too reliant on Claude or something. Maybe that's true, but I'd feel the same about the prospect of loosing ripgrep for a…
> If I do hit the limit, that's it for the entire week—a long time to be without a tool I've grown accustomed to!
These limits will only get worse. Let this be a wake-up call for you to not put all your development eggs in the A.I. basket.
> If I use claude code & claude its over in about 30 minutes. Can you share what you're doing? I've been experimenting with Claude Code and I feel like I have to be doing a lot with it before I even start seeing the usage warning limits on the $20/month plan. When I see people claiming they're getting rate limited after 30 minutes on the $100/month plan I have a hard time understanding what they're doing so different…
As a $20 month user, I can tell you in my experience it's "refactoring" jobs that really smash through those tokens quickly. If you do a "write a component that does this" kinda thing, you can use the $20 plan almost an unlimited amount of time. If you are doing "find all instances of ComponentFoo, change to ComponentBar, refactor each screen for correct usage of ComponentBar" kinda things, it's going to grep through…
> If you are doing "find all instances of ComponentFoo, change to ComponentBar, refactor each screen for correct usage of ComponentBar" kinda things, it's going to grep through your code, find multiple files, read all of them into context and start making changes one by one and/or spin up a subagent to do it. You'll be rate limited pretty quick doing things that way.
Huh?? grep and sed do this for free; you don't need A.I. for that.
Ok I really really have to figure out how to have a local setup of the open-source LLMs. I know i know - the "fixed costs" are high. But I have a strong feeling being able to setup local LLMs (and the rig for it) is the next build-your-own-PC phase. All I want is a coding agent and the grunt power to run it locally. Everything else Il build (generate) with it. I see so many folks claiming crazy hardware rigs and perf…
You can build a decent rig for yourself with: - 2x 4070 Ti (32 GB total VRAM) - $2200 - 64 GB RAM - $200-250 - Core i9/Ryzen 9 CPU - $450 - 2 TB SSD - $150 - Motherboard, cooler, case, PSU - $500-600 Total - ~$3500-3700, say $4000 with extras.