11–20 of 72 posts
Outcome billing is ideal for pretty much any SaaS product. Sounds great in theory, until you realize everyone has a different definition of outcome.
Understood. Take for instance, customer support Agent , that is supposed to resolve tickets. Assuming it resolves around 30% tickets by an objective measure. Do you think that cannot be captured and agreed upon by both sides?
In many cases though, you don’t know whether the outcome is correct or not but we just have evals for that.
Our product is a SOTA recall-first web search for complex queries. For example, let’s say your agent needs to find all instances of product launches in the past week.
“Classic” web search would return top results while ours return a full dataset where each row is a unique product (with citations to web pages)
We charge a flat fee per record. So, if we found 100 records, you pay us for 100. Of its 0 then it’s free.
So who's the arbiter to determine if the outcome was achieved? And how do you programmatically measure it?
I'd assume an outcome is a negotiated agreement between buyer and Agent provider.
Think of all the n8n workflows. If we take a simple example of Expense receipt processing workflows, or a lead sourcing workflow, I'd think the outcomes can be counted pretty well. In these cases, successfully entered receipts into ERP or number of Entries captured in salesforce.
I am sure there are cases where outcomes are fuzzy, for instances employer-employee agreement.
But in some cases, for instance, my accounting agent would only get paid if he successfully uploads my tax returns.
Surely not applicable in all cases. But, in cases Where a human is measured on outcomes, the same should be applicable for agents too, I guess
I believe it was under British rule, they offered a reward for people bringing in dead cobras as proof of culling. Which worked until people started breeding them just to get the reward. Humans gamed the system and it made the problem worse.
You can apply the same philosophy to employees and if you dare to do so you will quickly find out that it does not work. When a measure becomes a target, it ceases to be a good measure - Goodhart's law. I cannot see why AI agents should be treated differently when it comes to fuzzy measurements of performance.
You can apply the same philosophy to employees and if you dare to do so you will quickly find out that it does not work. When a measure becomes a target, it ceases to be a good measure - Goodhart's law. I cannot see why AI agents should be treated differently when it comes to fuzzy measurements of performance.
Bcuz the performance is usually not fuzzy and also the law only applies to certain jobs -- you would not apply the law to salesmen or customer support agents.
Outcome billing is ideal for pretty much any SaaS product. Sounds great in theory, until you realize everyone has a different definition of outcome.
Understood. Take for instance, customer support Agent , that is supposed to resolve tickets. Assuming it resolves around 30% tickets by an objective measure. Do you think that cannot be captured and agreed upon by both sides?
Earlier quoted context omitted.
Understood. Take for instance, customer support Agent , that is supposed to resolve tickets. Assuming it resolves around 30% tickets by an objective measure. Do you think that cannot be captured and agreed upon by both sides?
You get what you measure. The bot might be really bad and customers close the chat and it gets counted as success etc.
The same oversight mechanism that applies to humans cannot correct the flaws of AI agents?
Earlier quoted context omitted.
Bcuz the performance is usually not fuzzy and also the law only applies to certain jobs -- you would not apply the law to salesmen or customer support agents.
Salesmen making bad deals that boost their numbers and then don't make money in the long-term is one of the first things you learn when you work in an org that sells in the enterprise market.