Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why outcome-billing makes sense for AI Agents

valmi.io

Post reply on HN
Print view View original

51–60 of 72 posts

Re: Why outcome-billing makes sense for AI Agents

#51

Post by ivanstojic »

altcognito wrote: post #39
ivanstojic wrote: post #34

I started reading the article and immediately got hit by the incorrect statement in the opening: > If AI agents help each support employee handle 30% more tickets, that's like adding 30 new hires to a 100-person team, without the cost. I think this is an oversimplification designed to make LLMs seem more profitable than they actually are.

This is an article written by a company/llm trying to justify huge increases to the pricing structure. Oh! Yknow that thing we were charging you $200 a month for now? We're going to start charging you for the value we provide, and it will now be $5,000 a month. Meanwhile, the metrics for "value" are completely gamed.

At the same time, I actually wouldn’t mind a world in which AI agents cost $5000 a month if that’s what companies want to charge.

I feel like at some level that would remove the possibility of making a “just as good as humans but basically free” arguments and move discussion in the direction that feels more productive: discussing real benefits and shortcomings of both. Eg, loss of context with agents vs HR costs with humans, etc…

Re: Why outcome-billing makes sense for AI Agents

#52

Post by Neywiny »

rajvarkala wrote: post #21
Neywiny wrote: post #14

Maybe it's not as nice a story there as he's from India, but outside India people like to talk about their cobra problem and failed solution (retold below). This feels like that. If it's a ticket system, it could close them all as unresovable overnight. If it cares about customer satisfaction, it could give everybody thousand dollar gift cards. Point is, AIs existence is predicated on finding a way to improve its sco…

Sure, incentives can be gamed. The same oversight mechanism that applies to humans cannot correct the flaws of AI agents? What do you think is the catch? I am not saying things are clearly defined in most settings. But my accounting agent ( real person) gets paid only when he files my tax returns.

I think it gets more nebulous. For example, does he only get paid if the tax returns are accepted by the government? If they aren't, he's still put in the work. This becomes an extremely slippery slope. A better example is probably retail. In the US at least, places like Walmart and Amazon allow for returns, but they usually just throw it out. That's gotta be built into the price. Meaning, the cheapy no returns accepted online stores are cheaper because the cost for the purchaser isn't tied to satisfaction.

Your accountant has to build in margin that you pay for for clients who stiff him on the bill or who he has to take to court to argue he did the service as described in the contract. If you didn't hold that threshold over his head, he would be able to charge less. Would he? Maybe not, I don't know the guy, but he could.

Re: Why outcome-billing makes sense for AI Agents

#53

Post by rajvarkala »

Neywiny wrote: post #52
rajvarkala wrote: post #21

Earlier quoted context omitted.

Sure, incentives can be gamed. The same oversight mechanism that applies to humans cannot correct the flaws of AI agents? What do you think is the catch? I am not saying things are clearly defined in most settings. But my accounting agent ( real person) gets paid only when he files my tax returns.

I think it gets more nebulous. For example, does he only get paid if the tax returns are accepted by the government? If they aren't, he's still put in the work. This becomes an extremely slippery slope. A better example is probably retail. In the US at least, places like Walmart and Amazon allow for returns, but they usually just throw it out. That's gotta be built into the price. Meaning, the cheapy no returns accep…

Understood. So, a better way is to keep him on a retainer? Or let Amazon or Cheaper store do a cost-plus model?

I think that is the core of the argument. It is the risk-sharing between buyer and seller. If sold on outcomes, seller carries all risk. If sold on work-put-in, buyer carries all risk.

Add to that, in some scenarios, outcomes themselves are fuzzy.

Re: Why outcome-billing makes sense for AI Agents

#54

Post by htrp »

rajvarkala wrote: post #13
alberth wrote: post #4

So who's the arbiter to determine if the outcome was achieved? And how do you programmatically measure it?

Hi alberth, I'd assume an outcome is a negotiated agreement between buyer and Agent provider. Think of all the n8n workflows. If we take a simple example of Expense receipt processing workflows, or a lead sourcing workflow, I'd think the outcomes can be counted pretty well. In these cases, successfully entered receipts into ERP or number of Entries captured in salesforce. I am sure there are cases where outcomes are…

> But in some cases, for instance, my accounting agent would only get paid if he successfully uploads my tax returns.

I think you'd want it to correctly compute your taxes. Especially if you get a letter a year or two after the fact saying you owe the government money

Re: Why outcome-billing makes sense for AI Agents

#55

Post by Neywiny »

rajvarkala wrote: post #53
Neywiny wrote: post #52

Earlier quoted context omitted.

I think it gets more nebulous. For example, does he only get paid if the tax returns are accepted by the government? If they aren't, he's still put in the work. This becomes an extremely slippery slope. A better example is probably retail. In the US at least, places like Walmart and Amazon allow for returns, but they usually just throw it out. That's gotta be built into the price. Meaning, the cheapy no returns accep…

Understood. So, a better way is to keep him on a retainer? Or let Amazon or Cheaper store do a cost-plus model? I think that is the core of the argument. It is the risk-sharing between buyer and seller. If sold on outcomes, seller carries all risk. If sold on work-put-in, buyer carries all risk. Add to that, in some scenarios, outcomes themselves are fuzzy.

Cost plus I'm not sure on. Maybe if your work was in small enough chunks. But if, did example, just generating one response is too expensive, there's no plus, it's just somebody paid $ for some bits in GPU memory and that's likely not useful to anyone.

Yes exactly. Your second paragraph hits the nail on the head. And I'm sure you agree that the AI companies aren't going to take on more risk for free.

Re: Why outcome-billing makes sense for AI Agents

#56

Post by _DeadFred_ »

ivanstojic wrote: post #34

I started reading the article and immediately got hit by the incorrect statement in the opening: > If AI agents help each support employee handle 30% more tickets, that's like adding 30 new hires to a 100-person team, without the cost. I think this is an oversimplification designed to make LLMs seem more profitable than they actually are.

If the AI does all the easy tickets, there's no easing in new hires, so that process is going to be more expensive, so I better get discounted for that hit.

If there is zero slack, and only the hardest parts, this is no longer the job it was before. Salaries will have to go up, or retention will go down. In addition these jobs could already be awful when there was some slack, removing all slack tasks to AI is going to make them miserable so average customer interaction once they get to a human agent is probably going to be worse so your customer satisfaction will take a hit. So I better get discounted with that reputational hit.

It's like the 'have AI pick the tomatoes it can, and the field worker the rest'. Picking the easy tomatoes is factored into the job. Having the ai pick the easy ones could break the whole model. Of having zero slack for the workers could break them and result in no one showing up to jobs where AI has done the easy picking.

Re: Why outcome-billing makes sense for AI Agents

#57

Post by HelloMcFly »

rajvarkala wrote: post #8
SkyPuncher wrote: post #6

Outcome billing is ideal for pretty much any SaaS product. Sounds great in theory, until you realize everyone has a different definition of outcome.

Understood. Take for instance, customer support Agent , that is supposed to resolve tickets. Assuming it resolves around 30% tickets by an objective measure. Do you think that cannot be captured and agreed upon by both sides?

At scale? Programmatically? In a way that actually saves time and doesn't create billing conflict (that always happens to benefit the LLM vendor)?

No I do not.

Re: Why outcome-billing makes sense for AI Agents

#58

Post by HelloMcFly »

throwaway__ai wrote: post #30
artembugara wrote: post #27

Earlier quoted context omitted.

Do you mean it’s written by AI? Or just my writing style?

Not the writing style, but the fact that the em-dashes and strange ticks make it indistinguishable from something AI-generated. At least take the time to replace them with something you can produce easily on a physical keyboard. Edit: Well, actually - this kind of writing style does feel quite AI-ish: > It really makes sense, and the best part — customers love it

The em dashes didn't strike me as LLM because they had spaces on either side, something I don't typically see in LLM outputs as much. But the quote you highlighted is pretty much dead-on for LLM "speak" I must admit. In the end though, I think this is human written.

Re: Why outcome-billing makes sense for AI Agents

#59

Post by rajvarkala »

HelloMcFly wrote: post #57
rajvarkala wrote: post #8

Earlier quoted context omitted.

Understood. Take for instance, customer support Agent , that is supposed to resolve tickets. Assuming it resolves around 30% tickets by an objective measure. Do you think that cannot be captured and agreed upon by both sides?

At scale? Programmatically? In a way that actually saves time and doesn't create billing conflict (that always happens to benefit the LLM vendor)? No I do not.

Interesting. Let's take the case of infra spend on AWS. Amazon says you invoked serverless calls 100k times and you are charged for it. How are you trusting them?

Re: Why outcome-billing makes sense for AI Agents

#60

Post by rajvarkala »

deathanatos wrote: post #18
rajvarkala wrote: post #8

Earlier quoted context omitted.

Understood. Take for instance, customer support Agent , that is supposed to resolve tickets. Assuming it resolves around 30% tickets by an objective measure. Do you think that cannot be captured and agreed upon by both sides?

Already, today, human customer support agents' performance is measured in ticket resolution, and the Goodhart's Law consequences of that are trivial visible to anyone that's ever tried to get a ticket actually resolved , as opposed to simply marked "resolved" in a ticketing system somewhere…

We just give today's human performance metrics to AI agents.

AI agent developers internally have a metric they are targeting to improve. That itself violates goodhart law.

Post reply on HN