Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How are people forecasting AI API costs for agent workflows?

news.ycombinator.com

Post reply on HN
Print view View original

1–10 of 28 posts

Ask HN: How are people forecasting AI API costs for agent workflows?

#1

Post by Barathkanna »

I’ve been experimenting with agent-based features and one thing that surprised me is how hard it is to estimate API costs.

A single user action can trigger anywhere from a few to dozens of LLM calls (tool use, retries, reasoning steps), and with token-based pricing the cost can vary a lot.

How are builders here planning for this when pricing their SaaS?

Are you just padding margins, limiting usage, or building internal cost tracking? Also curious, would a service that offers predictable pricing for AI APIs (like a fixed subscription cost) actually be useful for people building agentic workflows?

Re: Ask HN: How are people forecasting AI API costs for agent workflows?

#4

Post by Lazy_Player82 »

Honestly, if you're designing your agent workflows properly with hard limits on retries and tool calls, the variance shouldn't be that wild. Most of the unpredictability comes from not having those guardrails in place early on. A few weeks of real production data usually shows the average cost is more stable than you'd expect.

Re: Ask HN: How are people forecasting AI API costs for agent workflows?

#5

Post by Barathkanna »

Lazy_Player82 wrote: post #4

Honestly, if you're designing your agent workflows properly with hard limits on retries and tool calls, the variance shouldn't be that wild. Most of the unpredictability comes from not having those guardrails in place early on. A few weeks of real production data usually shows the average cost is more stable than you'd expect.

True, but for early stage builders it’s harder to design those guardrails upfront. A lot of the time you only discover the retry patterns and cost spikes once real users start hitting the system.

Re: Ask HN: How are people forecasting AI API costs for agent workflows?

#6

Post by Lazy_Player82 »

Barathkanna wrote: post #5
Lazy_Player82 wrote: post #4

Honestly, if you're designing your agent workflows properly with hard limits on retries and tool calls, the variance shouldn't be that wild. Most of the unpredictability comes from not having those guardrails in place early on. A few weeks of real production data usually shows the average cost is more stable than you'd expect.

True, but for early stage builders it’s harder to design those guardrails upfront. A lot of the time you only discover the retry patterns and cost spikes once real users start hitting the system.

Fair point. And honestly, with more non-technical builders shipping agent-based products these days, that's probably where a service like this makes the most sense – for people who don't yet have the experience to know what guardrails to put in place.

Re: Ask HN: How are people forecasting AI API costs for agent workflows?

#7

Post by Barathkanna »

Lazy_Player82 wrote: post #6
Barathkanna wrote: post #5

Earlier quoted context omitted.

True, but for early stage builders it’s harder to design those guardrails upfront. A lot of the time you only discover the retry patterns and cost spikes once real users start hitting the system.

Fair point. And honestly, with more non-technical builders shipping agent-based products these days, that's probably where a service like this makes the most sense – for people who don't yet have the experience to know what guardrails to put in place.

Exactly. That’s actually why we started building Oxlo.ai. Early stage builders usually just want to experiment without worrying too much about token cost spikes.

Re: Ask HN: How are people forecasting AI API costs for agent workflows?

#9

Post by clearloop »

Barathkanna wrote: post #3
clearloop wrote: post #2

imo switch to local models could be an option

Local models solve the marginal cost problem, but they move the complexity into infrastructure and throughput planning instead.

makes sense, it really depends on the use cases, I'm building my version of claw openwalrus for the local LLMs first goal, I think myself will use local models for daily tasks that heavily depend on tool callings, but for coding or doing research, I'll keep using remote models

and this topic actually inspires me that I can introduce a builtin gas meter for tokens

Post reply on HN