Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Agents need control flow, not more prompts

bsuh.bearblog.dev

Post reply on HN
Print view View original

121–130 of 348 posts

Re: Agents need control flow, not more prompts

#121

Post by pishpash »

cluckindan wrote: post #115
khasan222 wrote: post #107

Earlier quoted context omitted.

Completely agree! People tend to forget we are non deterministic too! Yet we are able to write code fine, and fairly reliably by using tools that can help keep us fairly honest. I think most problems with ai tend to be around can you deterministically test the thing you are asking it to do? How many of us would never ever show work, without going to check the thing we just built first?

> can you deterministically test the thing you are asking it to do? Of course: have it write tests first; and run them to check its work. Works well for refactoring, but greenfield implementations still rely on a spec that is guaranteed to be incomplete, overcomplete and wrong in many ways.

You can't ask something to check its own work without external reward/penalty. It'll cheat.

Re: Agents need control flow, not more prompts

#122

Post by Nizoss »

tim-projects wrote: post #33

This is exactly the problem I've been working on and I see others are too. When you implement quality control gates, everything works better. It solves so many of the basic problems llms create - saying code is finished when it isn't. Skipping tests, introducing code regressions, basic code validation etc I am finding that the better the quality gates are the lower quality llm you can use for the same result (at a co…

Exactly! I don’t babysit TDD anymore. I have another agent that does that for me and honestly sometimes catches things I would have missed if I was the babysitting.

Hooks do wonders here. The payload contains a lot of information about the pending action the agent wants to make. Combine that with the most recent n events from the agent’s session history and you have a rich enough context to pass to another agent to validate the action through the SDK.

This way the validation uses the same subscription you’re logged in to, whether you’re using Claude Code, Codex, or Copilot. The validation agent responds with a json format that you can easily parse and return, allowing you to let the action through or block it with direction and guidance. I’m genuinely impressed by how well this works considering how simple it is.

You can find my approach here: https://github.com/nizos/probity

Re: Agents need control flow, not more prompts

#123

Post by aleksiy123 »

jmaw wrote: post #55
aleksiy123 wrote: post #35

Earlier quoted context omitted.

It’s already here the question is just to what extent? Are google search results modifying your software at runtime? Take or agent chat for example, the output text is a ui, agents can generate charts and even constrained ui elements. Isn’t that created and adapted at run time? If you mean like agents live modifying your code. I think that’s pretty much here as well. Can read the logs and send prs. The only thing is…

Some kind of "code", you could say

Yes but more declarative vs imperative.

I say what the llm says how.

Re: Agents need control flow, not more prompts

#124

Post by pishpash »

aleksiy123 wrote: post #35
edgarvaldes wrote: post #16

Earlier quoted context omitted.

Some have expressed the opinion in this forum that the future of software lies in programs that are created and adapted at runtime, using genAI. I don't know how far we are from that.

It’s already here the question is just to what extent? Are google search results modifying your software at runtime? Take or agent chat for example, the output text is a ui, agents can generate charts and even constrained ui elements. Isn’t that created and adapted at run time? If you mean like agents live modifying your code. I think that’s pretty much here as well. Can read the logs and send prs. The only thing is…

Sounds like assemblers bemoaning loss of control to C. The solution was inline assembly...

Re: Agents need control flow, not more prompts

#125

Post by svachalek »

pishpash wrote: post #118
827a wrote: post #100

1000% agree. I am increasingly hesitant to believe Anthropic's continual war drum of "build for the capabilities of future models, they'll get better". We've got a QA agent that needs to run through, say, 200 markdown files of requirements in a browser session. Its a cool system that has really helped improve our team's efficiency. For the longest time we tried everything to get a prompt like the following working: "…

Can you not have it write your harness for you, or have it be the first step? You can push your own determinism where you need, surely.

True. The prompt reads: Run the following Python: ```

Re: Agents need control flow, not more prompts

#126

Post by woeirua »

827a wrote: post #100

1000% agree. I am increasingly hesitant to believe Anthropic's continual war drum of "build for the capabilities of future models, they'll get better". We've got a QA agent that needs to run through, say, 200 markdown files of requirements in a browser session. Its a cool system that has really helped improve our team's efficiency. For the longest time we tried everything to get a prompt like the following working: "…

I have but one upvote, but yes. The only way to make these systems work reliably is to break the problems down into smaller chunks. Any internal consistency checks are just going to show you that LLMs are way less consistent than you’d expect.

Re: Agents need control flow, not more prompts

#127

Post by pishpash »

aleksiy123 wrote: post #123
jmaw wrote: post #55

Earlier quoted context omitted.

Some kind of "code", you could say

Yes but more declarative vs imperative. I say what the llm says how.

Not that long ago the workflow was to turn code comments into code. Maybe leave some comments as is now.

Re: Agents need control flow, not more prompts

#130

Post by QuercusMax »

xuhu wrote: post #41

It sounds like the "app written in C++ calling Lua scripts, versus app written in Lua calling C++ libraries" debate. Both designs (Lightroom, game engines) have worked successfully. There's probably nothing that prevents mixing both approaches in the same "app".

This pattern has been described for decades: https://wiki.c2.com/?AlternateHardAndSoftLayers. It's not just a matter of who's in control - you can layer these things.
Post reply on HN