Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is there a half-life for the success rates of AI agents?

tobyord.com

Post reply on HN
Print view View original

81–90 of 143 posts

Re: Is there a half-life for the success rates of AI agents?

#81

Post by eplatzek »

Workaccount2 wrote: post #16
mikeocool wrote: post #2

This very much aligns with my experience — I had a case yesterday where opus was trying to do something with a library, and it encountered a build error. Rather than fix the error, it decided to switch to another library. It then encountered another error and decided to switch back to the first library. I don’t think I’ve encountered a case where I’ve just let the LLM churn for more than a few minutes and gotten a go…

They poison their own context. Maybe you can call it context rot, where as context grows and especially if it grows with lots of distractions and dead ends, the output quality falls off rapidly. Even with good context the rot will start to become apparent around 100k tokens (with Gemini 2.5). They really need to figure out a way to delete or "forget" prior context, so the user or even the model can go back and prune…

Honestly that feels a like a human.

After hitting my head against a wall with a problem I need to stop.

I need to stop and clear my context. Go a walk. Talk with friends. Switch to another task.

Re: Is there a half-life for the success rates of AI agents?

#82

Post by insane_dreamer »

mikeocool wrote: post #2

This very much aligns with my experience — I had a case yesterday where opus was trying to do something with a library, and it encountered a build error. Rather than fix the error, it decided to switch to another library. It then encountered another error and decided to switch back to the first library. I don’t think I’ve encountered a case where I’ve just let the LLM churn for more than a few minutes and gotten a go…

I have the same experience. When it starts going down the wrong path, it won't switch paths. I have to intervene, put my thinking cap on, and tell the agent to start over from scratch and explore another path (which I usually have to define to get them started in the right direction). In the end, I'm not sure how much time I've actually saved over doing it myself.

Re: Is there a half-life for the success rates of AI agents?

#83

Post by alganet »

peacebeard wrote: post #73
alganet wrote: post #38

Earlier quoted context omitted.

> People are not usually being too specific about what they were trying to do. The same goes for a lot of programming discussion of course. In programming, I already have a very good tool to follow specific steps: _the programming language_. It is designed to run algorithms. If I need to be specific, that's the tool to use. It does exactly what I ask it to do. When it fails, it's my fault. Some humans require algorit…

I am not saying that people were not specific in their instructions to the LLM, but rather that in the discussion they are not sharing specific details of their success stories or failures. We are left seeing lots of people saying "it worked for me" and "it didn't work for me" without enough information to assess what was different in those cases. What I'm contending is that the essential differences in the challenge…

> they are not sharing specific details of their success stories or failures

Can you blame them for that?

For other products, do you think people contact customer support with an abundance of information?

Now, consider what these LLM products promise to deliver. Text box, answer. Is there any indication that different challenges might yield difference in the quality of outcome? Nope. Magic genie interface, it either works or it doesn't.

Re: Is there a half-life for the success rates of AI agents?

#85

Post by kennyadam »

PaulHoule wrote: post #39
bwfan123 wrote: post #25

Earlier quoted context omitted.

> It affects people too. Something I learned halfway through a theoretical physics PhD in the 1990s was that a 50-page paper with a complex calculation almost certainly had a serious mistake in it that you'd find if you went over it line-by-line. Interesting, and I used to think that math and sciences were invented by humans to model the world in a manner to avoid errors due to chains of fuzzy thinking. Also, formal…

These were symbolic calculations. Mine was a derivation of the Gutzwiller Trace Formula https://inspirehep.net/files/20b84db59eace6a7f90fc38516f530e... using integration over phase space instead of position or momentum space. Most people think you need an orthogonal basis set to do quantum mechanical calculation but it turns that "resolution of unity is all you need", that is, if you integrate |x> There are quite a f…

Exactly what I was going to say

Re: Is there a half-life for the success rates of AI agents?

#86

Post by no_wizard »

Workaccount2 wrote: post #16
mikeocool wrote: post #2

This very much aligns with my experience — I had a case yesterday where opus was trying to do something with a library, and it encountered a build error. Rather than fix the error, it decided to switch to another library. It then encountered another error and decided to switch back to the first library. I don’t think I’ve encountered a case where I’ve just let the LLM churn for more than a few minutes and gotten a go…

They poison their own context. Maybe you can call it context rot, where as context grows and especially if it grows with lots of distractions and dead ends, the output quality falls off rapidly. Even with good context the rot will start to become apparent around 100k tokens (with Gemini 2.5). They really need to figure out a way to delete or "forget" prior context, so the user or even the model can go back and prune…

I feel like as an end user I’d like to be able to do more to shape the LLM behavior. For example, I’d like to flag the dead end paths so they’re properly dropped out of context and not explored again, unless I as a user clear the flag(s).

I know there is work being done on LLM “memory” for lack of a better term but I have yet to see models get more responsive over time with this kind of feedback. I know I can flag it but right now it doesn’t help my “running” context that would be unique to me.

I have a similar thought about LLM “membranes”, which combines the learning from multiple users to become more useful, I am keeping a keen eye on that as I think that will make them more useful on a organizational level

Re: Is there a half-life for the success rates of AI agents?

#87

Post by drewnick »

libraryofbabel wrote: post #52
steveklabnik wrote: post #47

Earlier quoted context omitted.

> They really need to figure out a way to delete or "forget" prior context, so the user or even the model can go back and prune poisonous tokens. In Claude Code you can use /clear to clear context, or /compact to compact it down, with the message guiding what stays and what goes. It's helpful.

Also in Claude Code you can just press a bunch of times and you can backtrack to an earlier point in the history before the context was poisoned, and re-start from there. Claude has some amazing features like this that aren’t very well documented. Yesterday I just learned it writes sessions to disk and you can resume them where you left off with -continue or - resume if you accidentally close or something.

Thank you! This just saved me after closing laptop and losing a chat in VS Code. Cool feature and always a place where Clause Code UX was behind chat - being able to see history. "/continue" saved me ~15 minutes of re-establishing the planning for a new feature.

Also loving the shift + tab (twice) to enter plan mode. Just adding here in case it helps anyone else.

Re: Is there a half-life for the success rates of AI agents?

#88

Post by steveklabnik »

libraryofbabel wrote: post #52
steveklabnik wrote: post #47

Earlier quoted context omitted.

> They really need to figure out a way to delete or "forget" prior context, so the user or even the model can go back and prune poisonous tokens. In Claude Code you can use /clear to clear context, or /compact to compact it down, with the message guiding what stays and what goes. It's helpful.

Also in Claude Code you can just press a bunch of times and you can backtrack to an earlier point in the history before the context was poisoned, and re-start from there. Claude has some amazing features like this that aren’t very well documented. Yesterday I just learned it writes sessions to disk and you can resume them where you left off with -continue or - resume if you accidentally close or something.

> Claude has some amazing features like this that aren’t very well documented.

Yeah, it seems like they stealth ship a lot. Which is cool, but can sometimes lead to a future that's unevenly distributed, if you catch my drift.

Re: Is there a half-life for the success rates of AI agents?

#89

Post by potatolicious »

codeflo wrote: post #28
Workaccount2 wrote: post #16

Earlier quoted context omitted.

They poison their own context. Maybe you can call it context rot, where as context grows and especially if it grows with lots of distractions and dead ends, the output quality falls off rapidly. Even with good context the rot will start to become apparent around 100k tokens (with Gemini 2.5). They really need to figure out a way to delete or "forget" prior context, so the user or even the model can go back and prune…

I wonder to what extent this might be a case where the base model (the pure token prediction model without RLHF) is "taking over". This is a bit tongue-in-cheek, but if you see a chat protocol where an assistant makes 15 random wrong suggestions, the most likely continuation has to be yet another wrong suggestion. People have also been reporting that ChatGPT's new "memory" feature is poisoning their context. But cont…

I think this is at least somewhat true anecdotally. We do know that as context length increases, adherence to the system prompt decreases. Whether that de-adherence is reversion to the base model or not I'm not really qualified to say, but it certainly feels that way from observing the outputs.

Pure speculation on my part but it feels like this may be a major component of the recent stories of people being driven mad by ChatGPT - they have extremely long conversations with the chatbot where the outputs start seeming more like the "spicy autocomplete" fever dream creative writing of pre-RLHF models, which feeds and reinforces the user's delusions.

Many journalists have complained that they can't seem to replicate this kind of behavior in their own attempts, but maybe they just need a sufficiently long context window?

Re: Is there a half-life for the success rates of AI agents?

#90

Post by amlib »

vunderba wrote: post #79
matsemann wrote: post #74

Earlier quoted context omitted.

Reminds me of trying to train a small neural net to play Robocode ~10+ years ago. Tried to "punish" it for hitting walls, so next morning I had evolved a tanks that just stood still... Then punished it for standing still, ended up with a tanks just vibrating, alternating moving back and forth quickly, etc.

That's great. There's a pretty funny example of somebody training a neural net to play Tetris on the Nintendo entertainment system, and it quickly learned that if it was about to lose to just hit pause and leave the game in that state indefinitely.

I guess it came to the same conclusion as the computer in War Games, "The only way to win is not to play"
Post reply on HN