Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: CostPerPrompt – Live AI API pricing and real-workload cost calculators

costperprompt.com

Post reply on HN
Print view View original

1–10 of 18 posts

Re: Show HN: CostPerPrompt – Live AI API pricing and real-workload cost calculators

#4

Post by cortesoft »

How do you calculate the effect of caching?

Sometimes, I take breaks in the middle of a session and end up losing the prompt cache which drives up the token usage a ton. Don't all the providers have different cache times and behavior? If one person takes 20 minutes between messages, some services will keep that cache while some won't. Is there a way to factor that in?

Re: Show HN: CostPerPrompt – Live AI API pricing and real-workload cost calculators

#6

Post by schleck8 »

cortesoft wrote: post #4

How do you calculate the effect of caching? Sometimes, I take breaks in the middle of a session and end up losing the prompt cache which drives up the token usage a ton. Don't all the providers have different cache times and behavior? If one person takes 20 minutes between messages, some services will keep that cache while some won't. Is there a way to factor that in?

OpenRouter shows the cache hit rate and calculates an effective pricing per provider if you scroll down on the model page. Note that some providers run fp4 versions of models. DeepInfra often does this

Re: Show HN: CostPerPrompt – Live AI API pricing and real-workload cost calculators

#7

Post by ahmed_hassan7 »

cortesoft wrote: post #4

How do you calculate the effect of caching? Sometimes, I take breaks in the middle of a session and end up losing the prompt cache which drives up the token usage a ton. Don't all the providers have different cache times and behavior? If one person takes 20 minutes between messages, some services will keep that cache while some won't. Is there a way to factor that in?

Author here — great question, and you've hit the exact reason the calculator uses an adjustable cache-hit ratio (default ~70%) instead of assuming 100%: real hit rates depend on session gaps, and TTLs differ a lot by provider.

Rough current state: OpenAI's automatic caching typically keeps prefixes warm for ~5–60 min depending on load; Anthropic's default cache TTL is 5 min (extendable to 1h for an extra write fee); Gemini's explicit caches let you set your own TTL with per-hour storage pricing. So your 20-minute coffee break is usually fine on OpenAI, a cold cache on default Anthropic, and whatever you paid for on Gemini.

Adding an "average gap between messages" input that derates the hit rate per provider is a really good idea — putting it on the list. Thanks!

Re: Show HN: CostPerPrompt – Live AI API pricing and real-workload cost calculators

#8

Post by ahmed_hassan7 »

esafak wrote: post #3

It's easy to scrape the pricing; you have to do some leg work to estimate the token efficiency, which factors into the ultimate cost. Ideally you'd compare reasoning efficiency too; given them all the same task.

[flagged]
Post reply on HN