11–20 of 84 posts
You are charging per request ($0.05) what amounts to about 1M tokens on GPT4-mini ($0.075); i.e. this is 10-100x more expensive than the cost of running the LLM. I don't think it's worth it at that price point.
Edit (just found out this):
"The $0.05 per request fee covers requests up to 30,000 tokens.
For requests exceeding this limit, additional nominal charges may apply."
Hmm ... it's just too much.
One of the easiest setups I've ever had. Great documentation, getting started, and love the robustness and the speed of LLM responses.
This is actually really cool. I could see this being very useful. I can see big companies with a budget paying for this rather than rolling their own, but I would assume the biggest struggle you'll have with the long tail of users is pricing, since they may make their own reliability solutions that are "good enough." What's your take on that?
The pricing is ... steep? You are charging per request ($0.05) what amounts to about 1M tokens on GPT4-mini ($0.075); i.e. this is 10-100x more expensive than the cost of running the LLM. I don't think it's worth it at that price point. Edit (just found out this): "The $0.05 per request fee covers requests up to 30,000 tokens. For requests exceeding this limit, additional nominal charges may apply." Hmm ... it's just…
How are expectations specified?
The pricing is ... steep? You are charging per request ($0.05) what amounts to about 1M tokens on GPT4-mini ($0.075); i.e. this is 10-100x more expensive than the cost of running the LLM. I don't think it's worth it at that price point. Edit (just found out this): "The $0.05 per request fee covers requests up to 30,000 tokens. For requests exceeding this limit, additional nominal charges may apply." Hmm ... it's just…
The pricing is ... steep? You are charging per request ($0.05) what amounts to about 1M tokens on GPT4-mini ($0.075); i.e. this is 10-100x more expensive than the cost of running the LLM. I don't think it's worth it at that price point. Edit (just found out this): "The $0.05 per request fee covers requests up to 30,000 tokens. For requests exceeding this limit, additional nominal charges may apply." Hmm ... it's just…
Good feedback, I agree that our pay-as-you-go pricing may not fit everyone's budget. We're working on reducing our costs and simplifying our pricing. Goal is to get this much, much lower in the coming months. There's some complexity to how we handle pricing, since the core of what we do is based off of ingesting the requests and responses, and performing some processes on top of that info. Some people have small prom…
And the quality of your product is really good, so congrats on shipping, as usual!
I presume that's the reason for the limited selection of models - i.e. only some are tunable? I think that's my biggest issue with this solution, if I'm going to be dropping this into my LLM pipeline I need to have more control over how the model is used.
Good problem to take on, you're absolutely right that there is a lot of demand for reliability. Curious how effective the learning + tuning really is. I presume that's the reason for the limited selection of models - i.e. only some are tunable? I think that's my biggest issue with this solution, if I'm going to be dropping this into my LLM pipeline I need to have more control over how the model is used.