Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why outcome-billing makes sense for AI Agents

valmi.io

Post reply on HN
Print view View original

21–30 of 72 posts

Re: Why outcome-billing makes sense for AI Agents

#21

Post by rajvarkala »

Neywiny wrote: post #14

Maybe it's not as nice a story there as he's from India, but outside India people like to talk about their cobra problem and failed solution (retold below). This feels like that. If it's a ticket system, it could close them all as unresovable overnight. If it cares about customer satisfaction, it could give everybody thousand dollar gift cards. Point is, AIs existence is predicated on finding a way to improve its sco…

Sure, incentives can be gamed.

The same oversight mechanism that applies to humans cannot correct the flaws of AI agents? What do you think is the catch?

I am not saying things are clearly defined in most settings. But my accounting agent ( real person) gets paid only when he files my tax returns.

Re: Why outcome-billing makes sense for AI Agents

#22

Post by _pdp_ »

wagwang wrote: post #16
_pdp_ wrote: post #15

You can apply the same philosophy to employees and if you dare to do so you will quickly find out that it does not work. When a measure becomes a target, it ceases to be a good measure - Goodhart's law. I cannot see why AI agents should be treated differently when it comes to fuzzy measurements of performance.

Bcuz the performance is usually not fuzzy and also the law only applies to certain jobs -- you would not apply the law to salesmen or customer support agents.

ok... how do you measure the performance of a coding assistant? Counting the lines of code written, bugs closed, PRs reviewed, some fuzzy measurement of quality or something else?

Re: Why outcome-billing makes sense for AI Agents

#23

Post by wagwang »

_pdp_ wrote: post #22
wagwang wrote: post #16

Earlier quoted context omitted.

Bcuz the performance is usually not fuzzy and also the law only applies to certain jobs -- you would not apply the law to salesmen or customer support agents.

ok... how do you measure the performance of a coding assistant? Counting the lines of code written, bugs closed, PRs reviewed, some fuzzy measurement of quality or something else?

I think this article is moreso referring to support and other rote processing-like agents.

Re: Why outcome-billing makes sense for AI Agents

#24

Post by higginsniggins »

SkyPuncher wrote: post #6

Outcome billing is ideal for pretty much any SaaS product. Sounds great in theory, until you realize everyone has a different definition of outcome.

If your customer base is so broud that you can't define a clear outcome for your nitche, your company probably isnt focused enough. Especially for a start up.

Re: Why outcome-billing makes sense for AI Agents

#26

Post by throwaway__ai »

artembugara wrote: post #12

It really makes sense, and the best part — customers love it. It’s the simple form of pricing, and it’s simple to understand. In many cases though, you don’t know whether the outcome is correct or not but we just have evals for that. Our product is a SOTA recall-first web search for complex queries. For example, let’s say your agent needs to find all instances of product launches in the past week. “Classic” web searc…

I get sad when I read comments like these, because I feel like HN is the only forum left where real discussion between real people providing real thoughts are happening. I think that is changing unfortunately. The em-dashes and the strange ticks immediate triggers my anti-bodies and devalues it, whether that is appropriate or not.

Re: Why outcome-billing makes sense for AI Agents

#27

Post by artembugara »

throwaway__ai wrote: post #26
artembugara wrote: post #12

It really makes sense, and the best part — customers love it. It’s the simple form of pricing, and it’s simple to understand. In many cases though, you don’t know whether the outcome is correct or not but we just have evals for that. Our product is a SOTA recall-first web search for complex queries. For example, let’s say your agent needs to find all instances of product launches in the past week. “Classic” web searc…

I get sad when I read comments like these, because I feel like HN is the only forum left where real discussion between real people providing real thoughts are happening. I think that is changing unfortunately. The em-dashes and the strange ticks immediate triggers my anti-bodies and devalues it, whether that is appropriate or not.

Do you mean it’s written by AI?

Or just my writing style?

Re: Why outcome-billing makes sense for AI Agents

#28

Post by andy99 »

Is this actually different from just guaranteeing some metrics? Like if you have a document processing “agent” that extracts fields from forms, you’d have an accuracy threshold and have some checks set up to verify this?

Does “outcome billing” amount to anything different?

Re: Why outcome-billing makes sense for AI Agents

#29

Post by free_bip »

rajvarkala wrote: post #21
Neywiny wrote: post #14

Maybe it's not as nice a story there as he's from India, but outside India people like to talk about their cobra problem and failed solution (retold below). This feels like that. If it's a ticket system, it could close them all as unresovable overnight. If it cares about customer satisfaction, it could give everybody thousand dollar gift cards. Point is, AIs existence is predicated on finding a way to improve its sco…

Sure, incentives can be gamed. The same oversight mechanism that applies to humans cannot correct the flaws of AI agents? What do you think is the catch? I am not saying things are clearly defined in most settings. But my accounting agent ( real person) gets paid only when he files my tax returns.

Right, it doesn't work the same for humans as it does AI agents.

If you finetune a model and it starts misbehaving, what are you going to do to it exactly? PIP it? Fire it? Of course not. AIs cannot be managed the same ways as humans (and I would argue that's for the best). Best you can do is try using a different model, but you have no guarantee that whatever issue your model has is actually solved in the new one.

Re: Why outcome-billing makes sense for AI Agents

#30

Post by throwaway__ai »

artembugara wrote: post #27
throwaway__ai wrote: post #26

Earlier quoted context omitted.

I get sad when I read comments like these, because I feel like HN is the only forum left where real discussion between real people providing real thoughts are happening. I think that is changing unfortunately. The em-dashes and the strange ticks immediate triggers my anti-bodies and devalues it, whether that is appropriate or not.

Do you mean it’s written by AI? Or just my writing style?

Not the writing style, but the fact that the em-dashes and strange ticks make it indistinguishable from something AI-generated. At least take the time to replace them with something you can produce easily on a physical keyboard.

Edit:

Well, actually - this kind of writing style does feel quite AI-ish:

> It really makes sense, and the best part — customers love it

Post reply on HN