Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is there a half-life for the success rates of AI agents?

tobyord.com

Post reply on HN
Print view View original

91–100 of 143 posts

Re: Is there a half-life for the success rates of AI agents?

#91

Post by kosh2 »

As long as LLMs have no true memory, this is expected. Think about the movie Memento. That is the experience for an LLM.

What could any human do with a context window of 10 minutes and no other memory? You could write yourself notes… but you might not see them because soon you won’t know they are there. So maybe tattoo them on your body…

You could likely do a lot of things. Just follow a recipe and cook. Drive to work. But could you drive to the hardware store and get some stuff you need to build that ikea furniture? Might be too much context.

I think solving memory is solving agi.

Re: Is there a half-life for the success rates of AI agents?

#92

Post by bogdan »

esafak wrote: post #35
prmph wrote: post #17

The amusing things LLMs do when they have been at a problem for some time and cannot fix it: - Removing problematic tests altogether - Making up libs - Providing a stub and asking you to fill in the code

If humans can say "The proof is left as an exercise for the reader", why can't LLMs :)

If I'm paying someone to solve a problem for me and they tell me part of it is left for me to sort out then I'm gonna be pretty unhappy with how I spent my money.

Re: Is there a half-life for the success rates of AI agents?

#93

Post by insane_dreamer »

vadansky wrote: post #23
mikeocool wrote: post #2

This very much aligns with my experience — I had a case yesterday where opus was trying to do something with a library, and it encountered a build error. Rather than fix the error, it decided to switch to another library. It then encountered another error and decided to switch back to the first library. I don’t think I’ve encountered a case where I’ve just let the LLM churn for more than a few minutes and gotten a go…

I had a particularly hard parsing problem so I setup a bunch of tests and let the LLM churn for a while and did something else. When I came back all the tests were passing! But as I ran it live a lot of cases were still failing. Turns out the LLM hardcoded the test values as “if (‘test value’) return ‘correct value’;”!

While I haven't run into this egregious of an offense, I have had LLMs either "fix" the unit test to pass with buggy code, or, conversely, "fix" the code to so that the test passes but now the code does something different than it should (because the unit test was wrong to start with).

Re: Is there a half-life for the success rates of AI agents?

#94

Post by artursapek »

ffsm8 wrote: post #40
vadansky wrote: post #23

Earlier quoted context omitted.

I had a particularly hard parsing problem so I setup a bunch of tests and let the LLM churn for a while and did something else. When I came back all the tests were passing! But as I ran it live a lot of cases were still failing. Turns out the LLM hardcoded the test values as “if (‘test value’) return ‘correct value’;”!

Missed opportunity for the LLM, could've just switched to Volkswagen CI https://github.com/auchenberg/volkswagen

lmfao

Re: Is there a half-life for the success rates of AI agents?

#95

Post by jappgar »

kridsdale1 wrote: post #56
PaulHoule wrote: post #13

This was always my mental model. If you have a process with N steps where your probability of getting a step right is p, your chance of success is pᶰ, or 0 as N → ∞. It affects people too. Something I learned halfway through a theoretical physics PhD in the 1990s was that a 50-page paper with a complex calculation almost certainly had a serious mistake in it that you'd find if you went over it line-by-line. I thought…

Human health follows this principle too. N is the LifeSpan. The steps taken are cell division. Eventually enough problems accumulate that it fails systemically. Sexual reproduction is context-clearing and starting over from ROM.

This is precisely why there is not and never will be a fountain of youth.

Sure, you could be cloned, but that wouldn't be you. The process of accumulating memories is also the process of aging with death being an inevitability.

Software is sort of like this too, hence rewrites.

Re: Is there a half-life for the success rates of AI agents?

#96

Post by smeeth »

mikeocool wrote: post #2

This very much aligns with my experience — I had a case yesterday where opus was trying to do something with a library, and it encountered a build error. Rather than fix the error, it decided to switch to another library. It then encountered another error and decided to switch back to the first library. I don’t think I’ve encountered a case where I’ve just let the LLM churn for more than a few minutes and gotten a go…

I suspect its something to do with the following:

When humans get stuck solving problems they often go out to acquire new information so they can better address the barrier they encountered. This is hard to replicate in a training environment, I bet its hard to let an agent search google without contaminating your training sample.

Re: Is there a half-life for the success rates of AI agents?

#97

Post by jappgar »

furyofantares wrote: post #64
deadbabe wrote: post #5

This is another reason why there’s no point in carefully constructing prompts and contexts trying to coax the right solution out of an LLM. The end result becomes more brittle with time. If you can’t zero shot your way to success the LLM simply doesn’t have enough training for your problem and you need a human touch or slightly different trigger words. There have been times where I’ve gotten a solution with such a mi…

This is half right, I think, and half very wrong. I always tell people if they're arguing with the LLM they're doing it wrong and for sure part of that is there's things they can't do and arguing won't change that. But the other part is it's hard to overstate their sensitivity to their context; when you're arguing about something it can do, you should start over with a better prompt (and, critically, no polluted cont…

Real arguments would be fine. The problem is that the LLM always acquiesces to your argument and then convinces itself that it will never do that again. It then proceeds to do it again.

I really think the insufferable obsequiousness of every LLM is one of the core flaws that make them terrible peer programmers.

So, you're right in a way. There's no sense in arguing with them, but only because they refuse to argue.

Re: Is there a half-life for the success rates of AI agents?

#98

Post by KronisLV »

onlyrealcuzzo wrote: post #10
mikeocool wrote: post #2

This very much aligns with my experience — I had a case yesterday where opus was trying to do something with a library, and it encountered a build error. Rather than fix the error, it decided to switch to another library. It then encountered another error and decided to switch back to the first library. I don’t think I’ve encountered a case where I’ve just let the LLM churn for more than a few minutes and gotten a go…

> If it doesn’t solve an issue on the first or second pass, it seems to rapidly start making things up, make totally unrelated changes claiming they’ll fix the issue, or trying the same thing over and over. Sounds like a lot of employees I know. Changing out the entire library is quite amusing, though. Just imagine: I couldn't fix this build error, so I migrated our entire database from Postgres to MongoDB...

That is amusing but I remember HikariCP in an older Java project having issues with DB connections against an Oracle instance. Thing is, no settings or debugging could really (easily) narrow it down, whereas switching to DBCP2 both fixed whatever was the stability issue in that particular pairing, as well as has nice abandoned connection tracking too. Definitely the quick and dirty solution that still has good results.

Re: Is there a half-life for the success rates of AI agents?

#99

Post by furyofantares »

jappgar wrote: post #97
furyofantares wrote: post #64

Earlier quoted context omitted.

This is half right, I think, and half very wrong. I always tell people if they're arguing with the LLM they're doing it wrong and for sure part of that is there's things they can't do and arguing won't change that. But the other part is it's hard to overstate their sensitivity to their context; when you're arguing about something it can do, you should start over with a better prompt (and, critically, no polluted cont…

Real arguments would be fine. The problem is that the LLM always acquiesces to your argument and then convinces itself that it will never do that again. It then proceeds to do it again. I really think the insufferable obsequiousness of every LLM is one of the core flaws that make them terrible peer programmers. So, you're right in a way. There's no sense in arguing with them, but only because they refuse to argue.

I've noticed I tend to word things in a way that implies the opposite of what I want it to. The obsequiousness is very obnoxious indeed.

But I think the bigger reason arguing doesn't work is they are still fundamentally next-token-predictors. The wrong answer was already something it thought was probable before it polluted its context with it. You arguing can be seen as an attempt to make the wrong answer less probable. But it already strengthened that probability by having already answered incorrectly.

Re: Is there a half-life for the success rates of AI agents?

#100

Post by Aeolun »

mikeocool wrote: post #2

This very much aligns with my experience — I had a case yesterday where opus was trying to do something with a library, and it encountered a build error. Rather than fix the error, it decided to switch to another library. It then encountered another error and decided to switch back to the first library. I don’t think I’ve encountered a case where I’ve just let the LLM churn for more than a few minutes and gotten a go…

I feel like they might eventually arrive at the right solution, but generally, interrupting it before it goes off on a wild tangent saves you quite a bit of time.
Post reply on HN