Does anyone else get dull feelings of dread reading this kind of thing? How do you combat it?
combat how? (And yes, yes I do)
We put a coding agent in a while loop
51–60 of 317 posts
Re: We put a coding agent in a while loop
#52Now I want to put one of these in a loop, give it access to some bitcoin, and tell it to come up with a viable strategy to become a billionaire within the next month.
Give it a spin
Re: We put a coding agent in a while loop
#53"At one point we tried “improving” the prompt with Claude’s help. It ballooned to 1,500 words. The agent immediately got slower and dumber. We went back to 103 words and it was back on track." Isn't this the exact opposite of every other piece of advice we have gotten in a year? Another general feedback just recently, someone said we need to generate 10 times, because one out of those will be "worth reviewing" How ca…
the core might be - the difference between an LLM context window, and an agent's orders in a text. LLM itself is a core engine, running in an environment of some kind (instruct vs others?). Agents on the other hand, are descendants of the old Marvin Minsky stuff in a way.. it has objectives and capacities, at a glance. LLMs are connected to modern agents because input text is read to start the agent.. inner loops are intermediate outputs of LLM, in language. There is no "internal code" to this set of agents, it is speaking in code and text to the next part of the internal process.
There are probably big oversights or errors in that short explanation. The LLM engine, the runner of the engine, and the specifics of some environment, make a lot of overlap and all of it is quite complicated.
hth
Re: We put a coding agent in a while loop
#54> After finishing the port, most of the agents settled for writing extra tests or continuously updating agent/TODO.md to clarify how "done" they were. In one instance, the agent actually used pkill to terminate itself after realizing it was stuck in an infinite loop. Ok, now that is funny! On so many levels. Now, for the project itself, a few thoughts: - this was tried before, about 1.5 years ago there was a project…
> - weird copyright / IP questions all around. This will be a minefield. Yeah, we're in weird territory because you can drive an LLM as a Bitcoin mixer over intellectual property. That's the entire point/meaning behind https://ghuntley.com/z80 . You can take something that exists, distill it back to specs, and then you've got your own IP. Throw away the tainted IP, and then just run Ralph over a loop. You are able to…
repoMirror is the wrong name, aiCodeLaundering would be more accurate. This is bulk machine translation from one language to another, but in this case, it is code.
Re: We put a coding agent in a while loop
#55Re: We put a coding agent in a while loop
#56Earlier quoted context omitted.
It's already happening brother, https://github.com/containers/kubernetes-mcp-server .
still don’t know why you need an MCP for this when the model is perfectly well trained to write files and run kubetctl on its own
Claude is, some models aren't. In some cases the MCPs do get the models to use tools better as well due to the schema, but I doubt kubectl is one of them (using the git mcp in claude code... facepalm)
Re: We put a coding agent in a while loop
#57> After finishing the port, most of the agents settled for writing extra tests or continuously updating agent/TODO.md to clarify how "done" they were. In one instance, the agent actually used pkill to terminate itself after realizing it was stuck in an infinite loop. Ok, now that is funny! On so many levels. Now, for the project itself, a few thoughts: - this was tried before, about 1.5 years ago there was a project…
> After finishing the port, most of the agents settled for writing extra tests or continuously updating agent/TODO.md to clarify how "done" they were. In one instance, the agent actually used pkill to terminate itself after realizing it was stuck in an infinite loop. Is that... the first recorded instance of an AI committing suicide?
I guess pkill would rather be a sleep or koma. Erasing itself from any storage would rather equate to aicide
Re: We put a coding agent in a while loop
#58> After finishing the port, most of the agents settled for writing extra tests or continuously updating agent/TODO.md to clarify how "done" they were. In one instance, the agent actually used pkill to terminate itself after realizing it was stuck in an infinite loop. Ok, now that is funny! On so many levels. Now, for the project itself, a few thoughts: - this was tried before, about 1.5 years ago there was a project…
> After finishing the port, most of the agents settled for writing extra tests or continuously updating agent/TODO.md to clarify how "done" they were. In one instance, the agent actually used pkill to terminate itself after realizing it was stuck in an infinite loop. Is that... the first recorded instance of an AI committing suicide?
The AI doesn't have a self preservation instinct. It's not trying to stay alive. There is usually an end token that means the LLM is done talking. There has been research on tuning how often that is emitted to shorten or lengthen conversations. The current systems respond well to RL for adjusting conversation length.
One of the providers (I think it was Anthropic) added some kind of token (or MCP tool?) for the AI to bail on the whole conversation as a safety measure. And it uses it to their liking, so clearly not trying to self preserve.
Re: We put a coding agent in a while loop
#59Earlier quoted context omitted.
still don’t know why you need an MCP for this when the model is perfectly well trained to write files and run kubetctl on its own
Claude is, some models aren't. In some cases the MCPs do get the models to use tools better as well due to the schema, but I doubt kubectl is one of them (using the git mcp in claude code... facepalm)
Yeah fair enough lol…usually I end up building model-optimized scripts instead of mcp which just flood context window with json and uuids (looking at you, linear) - much better to have Claude write 100 lines of ts to drop a markdown file with the issue and all comments and no noise
Re: We put a coding agent in a while loop
#60"At one point we tried “improving” the prompt with Claude’s help. It ballooned to 1,500 words. The agent immediately got slower and dumber. We went back to 103 words and it was back on track." Isn't this the exact opposite of every other piece of advice we have gotten in a year? Another general feedback just recently, someone said we need to generate 10 times, because one out of those will be "worth reviewing" How ca…
One of the big things I think a lot of tooling misses, which Geoffrey touches on is the automated feedback loops built into the tooling. I expect you could probably incorporate generation time and token cost to automatically self tune this over time. Perhaps such things as discovering which prompts and models are best for which tasks automatically instead of manually choosing these things.
You want to go meta-meta? Get ralph to spawn subagents that analyze the process of how feedback and experimentation with techniques works. Perhaps allocate 10% of the time and effort to identifying what's missing that would make the loops more effective (better context, better tooling, better feedback mechanism, better prompts, ...?). Have the tooling help produce actionable ideas for how humans in the loop can effectively help the tooling. Have the tooling produce information and guidelines for how to review the generated code.
I think one of the big things missing in many of the tools currently available is tracking metrics through the entire software development loop. How long does it take to implement a feature. How many mistakes were made? How many errors were caught by tests? How many tokens does it take? And then using this information to automatically self-tune.