131–140 of 143 posts
Is this just a context limitation, or are they missing some kind of self-correction loop? Curious if anyone has seen agents that can catch their own mistakes and adjust during a task. Would love to hear how far that has come.
This very much aligns with my experience — I had a case yesterday where opus was trying to do something with a library, and it encountered a build error. Rather than fix the error, it decided to switch to another library. It then encountered another error and decided to switch back to the first library. I don’t think I’ve encountered a case where I’ve just let the LLM churn for more than a few minutes and gotten a go…
They poison their own context. Maybe you can call it context rot, where as context grows and especially if it grows with lots of distractions and dead ends, the output quality falls off rapidly. Even with good context the rot will start to become apparent around 100k tokens (with Gemini 2.5). They really need to figure out a way to delete or "forget" prior context, so the user or even the model can go back and prune…
It mostly happens when you pass it similar but updated code, for some reason it then doesn't really see the newest version and reasons over obsolete content.
I've had one chat recover from this, though.
This very much aligns with my experience — I had a case yesterday where opus was trying to do something with a library, and it encountered a build error. Rather than fix the error, it decided to switch to another library. It then encountered another error and decided to switch back to the first library. I don’t think I’ve encountered a case where I’ve just let the LLM churn for more than a few minutes and gotten a go…
They poison their own context. Maybe you can call it context rot, where as context grows and especially if it grows with lots of distractions and dead ends, the output quality falls off rapidly. Even with good context the rot will start to become apparent around 100k tokens (with Gemini 2.5). They really need to figure out a way to delete or "forget" prior context, so the user or even the model can go back and prune…
This very much aligns with my experience — I had a case yesterday where opus was trying to do something with a library, and it encountered a build error. Rather than fix the error, it decided to switch to another library. It then encountered another error and decided to switch back to the first library. I don’t think I’ve encountered a case where I’ve just let the LLM churn for more than a few minutes and gotten a go…
They poison their own context. Maybe you can call it context rot, where as context grows and especially if it grows with lots of distractions and dead ends, the output quality falls off rapidly. Even with good context the rot will start to become apparent around 100k tokens (with Gemini 2.5). They really need to figure out a way to delete or "forget" prior context, so the user or even the model can go back and prune…
It's already the case on tools like block.github.io/goose:
```
Summarize Conversation This will summarize your conversation history to save context space.
Previous messages will remain visible but only the summary will be included in the active context for Goose. This is useful for long conversations that are approaching the context limit.
```
Speaking of the Kwa et al. paper, is there a site that updates the results as new LLMs come out?
Earlier quoted context omitted.
10,000 hours of what are we comparing ? . Compilers already do better than me million hours for every program I write because I am not crafting assembly code. Computers save billions of hours compared to doing it by hand with an abacus or pen and paper. Productivity of humans is always dependent on tools they have access to, even with agents become that much productive so will humans who use tools —— Projecting doubl…
Great writeup. There are other scaling axes of course, around data (even synthetic data) and improving AI generation at the 'software' layer (smarter design / training efficiencies / inference speed ups) — progress in those might make the the currently-unthinkable orders of magnitude $500b and beyond not as necessary?
I could be quite wrong of course but it is not a certain bet that we will get fundamental breakthroughs from them.
There are specific areas which are always going to have major improvements .
In the semi conductor industry, Low power processors or multi core dies etc produced some results when core innovations slowed down during 2008-2018, i.e. till before the current EUV breakthrough driven generations of chip advances.
The history of EUV lithography and ASML’s success is an unlikely tale and it happened after both public and industry consortium funding of work for 2 decades that was abandoned multiple times .
Breakthroughs will happen eventually, but each wave ( we are on the fourth one for AI?) stagnates after initial rapid progress .
Earlier quoted context omitted.
I ask it to build it to 3d voxel engine in Rust, and it just goes off and do it. Same for a vox file parser. Sure, it takes some creative prompting, and a lot of turns to get it to settle on the proper coordinate system for the whole thing, but it goes ahead and does it. This took me two days so far. Unfortunate, the scope of the thing is now so large that the quality rapidly starts to degrade.
Building something from scratch where there are plenty of examples public on github seems to be the easiest case. Put these agents on a real existing codebase and ask them to fix a bug and they become useless.
This very much aligns with my experience — I had a case yesterday where opus was trying to do something with a library, and it encountered a build error. Rather than fix the error, it decided to switch to another library. It then encountered another error and decided to switch back to the first library. I don’t think I’ve encountered a case where I’ve just let the LLM churn for more than a few minutes and gotten a go…
They poison their own context. Maybe you can call it context rot, where as context grows and especially if it grows with lots of distractions and dead ends, the output quality falls off rapidly. Even with good context the rot will start to become apparent around 100k tokens (with Gemini 2.5). They really need to figure out a way to delete or "forget" prior context, so the user or even the model can go back and prune…
But, OpenAI and friends should let me purge my questions and, more importantly, the LLM response from the chat. More often than not, it’s poisoning itself with bad ideas, flip-flopping, etc. I hate having to pick up and move to a new chat but if I don’t the conversation will only go downhill.
Earlier quoted context omitted.
when this happens I do thew following 1) switch to a more expensive llm and ask it to debug: add debugging statements, reason about what's going on, try small tasks, etc 2) find issue 3) ask it to summarize what was wrong and what to do differently next time 4) copy and paste that recommendation to a small text document 5) revert to the original state and ask the llm to make the change with the recommendation as cont…
This honestly sounds slower than just doing it myself, and with more potential for bugs or non-standard code. I've had the same experience as parent where LLMs are great for simple tasks but still fall down surprisingly quickly on anything complex and sometimes make simple problems complex. Just a few days ago I asked Claude how to do something with a library and rather than give me the simple answer it suggested I r…