Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is there a half-life for the success rates of AI agents?

tobyord.com

Post reply on HN
Print view View original

41–50 of 143 posts

Re: Is there a half-life for the success rates of AI agents?

#41

Post by bwfan123 »

PaulHoule wrote: post #39
bwfan123 wrote: post #25

Earlier quoted context omitted.

> It affects people too. Something I learned halfway through a theoretical physics PhD in the 1990s was that a 50-page paper with a complex calculation almost certainly had a serious mistake in it that you'd find if you went over it line-by-line. Interesting, and I used to think that math and sciences were invented by humans to model the world in a manner to avoid errors due to chains of fuzzy thinking. Also, formal…

These were symbolic calculations. Mine was a derivation of the Gutzwiller Trace Formula https://inspirehep.net/files/20b84db59eace6a7f90fc38516f530e... using integration over phase space instead of position or momentum space. Most people think you need an orthogonal basis set to do quantum mechanical calculation but it turns that "resolution of unity is all you need", that is, if you integrate |x> There are quite a f…

thanks !

Re: Is there a half-life for the success rates of AI agents?

#42

Post by vunderba »

HeWhoLurksLate wrote: post #33
Workaccount2 wrote: post #16

Earlier quoted context omitted.

They poison their own context. Maybe you can call it context rot, where as context grows and especially if it grows with lots of distractions and dead ends, the output quality falls off rapidly. Even with good context the rot will start to become apparent around 100k tokens (with Gemini 2.5). They really need to figure out a way to delete or "forget" prior context, so the user or even the model can go back and prune…

I've found issues like this happen extremely quickly with ChatGPT's image generation features - if I tell it to put a particular logo in, the first iteration looks okay, while anything after that starts to look more and more cursed / mutant.

gpt-image-1 is unfortunately particular vulnerable to this problem. The more you want to change the initial image - the better off you'd honestly be just starting an entirely new conversation.

Re: Is there a half-life for the success rates of AI agents?

#43

Post by vunderba »

vadansky wrote: post #23
mikeocool wrote: post #2

This very much aligns with my experience — I had a case yesterday where opus was trying to do something with a library, and it encountered a build error. Rather than fix the error, it decided to switch to another library. It then encountered another error and decided to switch back to the first library. I don’t think I’ve encountered a case where I’ve just let the LLM churn for more than a few minutes and gotten a go…

I had a particularly hard parsing problem so I setup a bunch of tests and let the LLM churn for a while and did something else. When I came back all the tests were passing! But as I ran it live a lot of cases were still failing. Turns out the LLM hardcoded the test values as “if (‘test value’) return ‘correct value’;”!

I've definitely seen this happen before too. Test-driven development isn't all that effective if the LLM's only stated goal is to pass the tests without thinking about the problem in a more holistic/contextual manner.

Re: Is there a half-life for the success rates of AI agents?

#44

Post by Davidzheng »

PaulHoule wrote: post #13

This was always my mental model. If you have a process with N steps where your probability of getting a step right is p, your chance of success is pᶰ, or 0 as N → ∞. It affects people too. Something I learned halfway through a theoretical physics PhD in the 1990s was that a 50-page paper with a complex calculation almost certainly had a serious mistake in it that you'd find if you went over it line-by-line. I thought…

It kind of reminds me of this vanishing gradient problem in ML early on, where really deep layers won't train b/c you get these gradients dying midway, and the solution was to add these bypass connections (resnets style). I wonder if you can have similar solutions. Ofc I think what happens in general is like control theory, like you should be able to detect going off-course with some probability too and correct [longer horizon you have probability of leaving the safe-zone so you still get the exp decay but in larger field]. Not sure how to connect all these ideas though.

Re: Is there a half-life for the success rates of AI agents?

#45

Post by EGreg »

ffsm8 wrote: post #40
vadansky wrote: post #23

Earlier quoted context omitted.

I had a particularly hard parsing problem so I setup a bunch of tests and let the LLM churn for a while and did something else. When I came back all the tests were passing! But as I ran it live a lot of cases were still failing. Turns out the LLM hardcoded the test values as “if (‘test value’) return ‘correct value’;”!

Missed opportunity for the LLM, could've just switched to Volkswagen CI https://github.com/auchenberg/volkswagen

This is gold lol

Re: Is there a half-life for the success rates of AI agents?

#46

Post by OtherShrezzing »

Workaccount2 wrote: post #16
mikeocool wrote: post #2

This very much aligns with my experience — I had a case yesterday where opus was trying to do something with a library, and it encountered a build error. Rather than fix the error, it decided to switch to another library. It then encountered another error and decided to switch back to the first library. I don’t think I’ve encountered a case where I’ve just let the LLM churn for more than a few minutes and gotten a go…

They poison their own context. Maybe you can call it context rot, where as context grows and especially if it grows with lots of distractions and dead ends, the output quality falls off rapidly. Even with good context the rot will start to become apparent around 100k tokens (with Gemini 2.5). They really need to figure out a way to delete or "forget" prior context, so the user or even the model can go back and prune…

>They really need to figure out a way to delete or "forget" prior context, so the user or even the model can go back and prune poisonous tokens.

This is possible in tools like LM Studio when running LLMs locally. It's a choice by the implementer to grant this ability to end users. You pass the entire context to the model in each turn of the conversation, so there's no technical reason stopping this feature existing, besides maybe some cost benefits to the inference vendor from cache.

Re: Is there a half-life for the success rates of AI agents?

#47

Post by steveklabnik »

Workaccount2 wrote: post #16
mikeocool wrote: post #2

This very much aligns with my experience — I had a case yesterday where opus was trying to do something with a library, and it encountered a build error. Rather than fix the error, it decided to switch to another library. It then encountered another error and decided to switch back to the first library. I don’t think I’ve encountered a case where I’ve just let the LLM churn for more than a few minutes and gotten a go…

They poison their own context. Maybe you can call it context rot, where as context grows and especially if it grows with lots of distractions and dead ends, the output quality falls off rapidly. Even with good context the rot will start to become apparent around 100k tokens (with Gemini 2.5). They really need to figure out a way to delete or "forget" prior context, so the user or even the model can go back and prune…

> They really need to figure out a way to delete or "forget" prior context, so the user or even the model can go back and prune poisonous tokens.

In Claude Code you can use /clear to clear context, or /compact to compact it down, with the message guiding what stays and what goes. It's helpful.

Re: Is there a half-life for the success rates of AI agents?

#48

Post by enraged_camel »

prmph wrote: post #15
enraged_camel wrote: post #8

Earlier quoted context omitted.

I've actually thought about this extensively, and experimented with various approaches. What I found is that the quality of results I get, and whether the AI gets stuck in the type of loop you describe, depends on two things: how detailed and thorough I am with what I tell it to do, and how robust the guard rails I put around it are. To get the best results, I make sure to give detailed specs of both the current situ…

But don't they keep forgetting the instructions after enough time have passed? How do you get around that? Do you add an instruction that after every action it should go back and read the instructions gain?

They do start "drifting" after a while, at which point I export the chat (using Cursor), then start a new chat and add the exported file and say "here's the previous conversation, let's continue where we left off". I find that it deals with the transition pretty well.

It's not often that I have to do this. As I mentioned in my post above, if I start the interaction with thorough instructions/specs, then the conversation concludes before the drift starts to happen.

Re: Is there a half-life for the success rates of AI agents?

#49

Post by ziml77 »

rurp wrote: post #12
qazxcvbnmlp wrote: post #7

Earlier quoted context omitted.

when this happens I do thew following 1) switch to a more expensive llm and ask it to debug: add debugging statements, reason about what's going on, try small tasks, etc 2) find issue 3) ask it to summarize what was wrong and what to do differently next time 4) copy and paste that recommendation to a small text document 5) revert to the original state and ask the llm to make the change with the recommendation as cont…

This honestly sounds slower than just doing it myself, and with more potential for bugs or non-standard code. I've had the same experience as parent where LLMs are great for simple tasks but still fall down surprisingly quickly on anything complex and sometimes make simple problems complex. Just a few days ago I asked Claude how to do something with a library and rather than give me the simple answer it suggested I r…

Yeah if it gets stuck and can't easily get itself unstuck, that's when I step in to do the work for it. Otherwise it will continue to make more and more of a mess as it iterates on its own code.

Re: Is there a half-life for the success rates of AI agents?

#50

Post by byyoung3 »

deadbabe wrote: post #5

This is another reason why there’s no point in carefully constructing prompts and contexts trying to coax the right solution out of an LLM. The end result becomes more brittle with time. If you can’t zero shot your way to success the LLM simply doesn’t have enough training for your problem and you need a human touch or slightly different trigger words. There have been times where I’ve gotten a solution with such a mi…

i think that this is a bit of an exageration, but i see what you are saying. Anything more than 4-5 re-prompts is diminishing.
Post reply on HN