[flagged]
Agents need control flow, not more prompts
21–30 of 348 posts
Re: Agents need control flow, not more prompts
#22I wonder if a part of the problem isn't just the misapplication of LLMs in the first place. As has been mentioned elsewhere, perhaps the agent's prompt should be to write code to accomplish as much of the task in as repeatable/verifiable/deterministic a way as possible. This would hopefully include validation of the agent's output as well. The overall goal would be to keep the LLM out of doing processing that could be more efficiently (and often correctly) handled programmatically.
Re: Agents need control flow, not more prompts
#23Earlier quoted context omitted.
Some have expressed the opinion in this forum that the future of software lies in programs that are created and adapted at runtime, using genAI. I don't know how far we are from that.
> Some have expressed the opinion in this forum that the future of software lies in programs that are created and adapted at runtime, using genAI. Good luck with that. Users will flood you with complaints if a button moves 5px to the left after a design update. A program that is generated at runtime, with not just a variable UI but also UX and workflows, would get you death threats.
I think many software adjacent folks are super excited because they can now have the personalized toothbrush they keep asking people to make for them.
The problem is that outside of that most people want boring and regular interfaces so they can get in and solve the problem and get out - they don't want to "love" it or care if its "sexy" they want it to work and get out of the way.
LLMs transmogrifying your software at ever request assumes people are software architects and creators who love the computer interface, and that just doesn't describe the bulk of the population.
Most people using computers use the to consume things or utilize access to things, not for their own sake, and they certainly don't think "what if I just had code to do x..." unless x is make them a lot of money.
Re: Agents need control flow, not more prompts
#24Sometimes it feels like Agents are just reinventing microservices. Except they are are doing it in the most inefficient way possible. It is certainly a good way for the LLM companies to sell more tokens
Re: Agents need control flow, not more prompts
#25this is just advocating for a harness, which has been the focus (along with evals) for at least the last three months by pretty much anyone working with agents professionally or seriously
Re: Agents need control flow, not more prompts
#26This was basically my realization as well. We are trying to get LLMs to write software the way humans do it, but they have a different set of strength and weaknesses. Structuring tooling around what LLMs actually do well seems like an obvious thing to do. I wrote about this in some detail here:
Re: Agents need control flow, not more prompts
#27This is straight outta 2023:
Agents aren't reliable; use workflows instead.
Re: Agents need control flow, not more prompts
#28Agents are probabilistic systems. A common mechanism to get a reliable answer from systems that can have variable output is to run them several times (ideally in separate, isolated instances) and then have something vote on the best result or use the most common result. This happens in things like rockets and aviation where you have multiple systems giving an answer and an orchestrator picking the result. I've tried…
But then, if an agent picks the best response, how would you know that that is reliable?
Re: Agents need control flow, not more prompts
#29If you're trying to get reliability and determinism out of the LLM, you've already lost
There’s a whole range between completely random and completely rule based deterministic.
Somewhere in between that I guess is the varying levels of intelligence more likely able to make the “right” decision for anything you throw at it.
Re: Agents need control flow, not more prompts
#30You can get a lot done with agentic programming without going "all in" on a gastown-like system, but I think there is a minimum viable setup: 1. an adversarial agent harness that uses one agent to create a plan and implement it, and another to review the plan and code-review each step. 2. an agentic validation suite -- a more flexible take on e2e testing. 3. some custom skills that explain how to use both of those fl…
Claude already creates multiple agents for some projects just to keep context windows smaller. I don't think it'll be long before they offer a testing agent along with their planning agent.