[flagged]
Show HN: Cost.dev (YC W21) – making agents cost-aware and cheaper to call
21–30 of 41 posts
Re: Show HN: Cost.dev (YC W21) – making agents cost-aware and cheaper to call
#22[flagged]
Re: Show HN: Cost.dev (YC W21) – making agents cost-aware and cheaper to call
#23[dead]
Re: Show HN: Cost.dev (YC W21) – making agents cost-aware and cheaper to call
#24[flagged]
Re: Show HN: Cost.dev (YC W21) – making agents cost-aware and cheaper to call
#25Earlier quoted context omitted.
We do cache the results locally so that we're not repeatedly hitting our pricing API. The LLM doesn't access that cache directly though as it'd suffer the token tax you mention. Instead we optimised our CLI to return agent optimised results. We're constantly iterating and improving on it, but it already reduces the tokens usage very significantly. I wrote about it here: https://www.infracost.io/resources/blog/we-cut-…
Great, will it be possible to see it in your profile?
I'm not sure I follow, which profile do you mean? My profile on HN?
I don't know if we'll keep dissecting every incremental improvement we make as (so far) the general approach is the same as documented in the existing blog post: document common use cases -> benchmark them -> identify bottlenecks/expensive hot spots -> fix them -> repeat
The main thing changing right now is observing new more frequent use cases (either because we're adding new capabilities, or users are doing things we didn't entirely predict) and adding them to the test cases.
Re: Show HN: Cost.dev (YC W21) – making agents cost-aware and cheaper to call
#26Earlier quoted context omitted.
Seems to be targeted at quickly reducing infa cost for small-human teams with high-compute costs. I can see some value, but it's something I'd want to review quarterly instead of per-commit. I might feel different if I was really trying to stretch some runway. I can see why YC is interested in this issue, as I'm sure lots of startups are trying to stretch that runway.
When we started, we thought everyone could use it from startups to medium sized companies. What we learnt is that the most value comes for the enterprises. The reason is they have used Terraform to decentralize the infra provisioning, so now instead of a central platform team making all the IaC changes, you have hundreds or thousands of engineers making changes every month. Each of them are making a lot of decisions…
If you're primarily serving enterprise then the $250/mo foot in the door price makes sense. No reason to make too aggressive of a play for the small market/mid market.
Re: Show HN: Cost.dev (YC W21) – making agents cost-aware and cheaper to call
#27Tokens can get expended very quickly driving costs up substantially since there is no clarity in its usage If your service can cut the use substantially, for a large company with a huge AI bill, the savings could make up for the cost of the service which I think is pricey in my opinion. But I can see how it could work out.
Re: Show HN: Cost.dev (YC W21) – making agents cost-aware and cheaper to call
#28Tokens can get expended very quickly driving costs up substantially since there is no clarity in its usage If your service can cut the use substantially, for a large company with a huge AI bill, the savings could make up for the cost of the service which I think is pricey in my opinion. But I can see how it could work out.
yep, the ROI for cloud and AI spend tools is fairly easy to measure: it has to save multiples of what it costs.
Re: Show HN: Cost.dev (YC W21) – making agents cost-aware and cheaper to call
#29This is an interesting approach. I think people still underestimate how quickly token limits and context bloat become the bottleneck when you start running agents in production loops. Seems very relevant topic also given the environment. Will check it out
Thanks - indeed, it's also difficult to estimate how much the agents will cost in advance, and how much changes could reduce or increase costs.
Re: Show HN: Cost.dev (YC W21) – making agents cost-aware and cheaper to call
#30[flagged]