Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Agents need control flow, not more prompts

bsuh.bearblog.dev

Post reply on HN
Print view View original

261–270 of 348 posts

Re: Agents need control flow, not more prompts

#261

Post by user34283 »

groovetandon wrote: post #110
rnxrx wrote: post #22

I wonder if a part of the problem isn't just the misapplication of LLMs in the first place. As has been mentioned elsewhere, perhaps the agent's prompt should be to write code to accomplish as much of the task in as repeatable/verifiable/deterministic a way as possible. This would hopefully include validation of the agent's output as well. The overall goal would be to keep the LLM out of doing processing that could b…

This is so true have been working on a project for exactly this principle - https://www.decisional.com/blog/workflow-automation-should-b... I think there is a fundamental incentive problem - code + llm + harness is bound to be more efficient but the labs want you to burn tokens so they are not going to tell you to use the code, just burn more tokens. They are asking us to forget about the token cost and reliability f…

Skills are not fairy dust but a combination of prompts and deterministic code, so that you get the best of both worlds.

Eg. Loop in the code, process the subagents non-deterministic response for the individual task.

This takes 10 minutes to set up, you just need to run something like /skill-builder and describe the desired workflow.

I imagine many people just don’t know that it’s possible. I only discovered it a few days ago myself.

It worked on the first try.

Re: Agents need control flow, not more prompts

#263

Post by throawayonthe »

827a wrote: post #100

1000% agree. I am increasingly hesitant to believe Anthropic's continual war drum of "build for the capabilities of future models, they'll get better". We've got a QA agent that needs to run through, say, 200 markdown files of requirements in a browser session. Its a cool system that has really helped improve our team's efficiency. For the longest time we tried everything to get a prompt like the following working: "…

> But, its also made the agent impossible to run on any managed agent platform (Cursor Cloud Agents, Anthropic, etc)

couldn't you "just" have it orchestrate a bunch of subagents? a la the superpower skill

definitely a worse solution, non deterministic orchestration + way higher token usage (unless there's a way to hide the subagent output from the orchestrator agent? i haven't used any of these platforms) but could work in some cases

Re: Agents need control flow, not more prompts

#264

Post by throawayonthe »

krzyk wrote: post #258
fny wrote: post #229

Earlier quoted context omitted.

Secret: "compile" that orchestration prompt. Determinism is solved by turning prompts into code that can in turn run agents or run code or both. Everyone misses this pattern with skills: you can just drop code alongside a SKILL.md to guarantee certain behaviors, but for some reason everyone's addicted to writing prompts. You don't even need to build a CLI. A simple skill.py with tasks does it. You can even have helpe…

Could you elaborate what does "compiling orchestration prompt" mean?

a guess but i think they mean take the orchestration prompt and prompt yet another llm to turn that prompt into code..?

Re: Agents need control flow, not more prompts

#265

Post by user34283 »

dkersten wrote: post #251

This is something I realised late last year while using Claude Code. The LLM shouldn't be the one in control of the workflow, because the LLM can make mistakes, skip steps, hallucinate steps, etc. Its also wasteful of tokens. I'm a firm believer that a "thin harness" is the wrong approach for this reason and that workflows should be enforced in code. Doing that allows you to make sure that the workflow is always foll…

I‘ve recently started to use skills and so far it’s been working great.

Your agent can write a python script to loop and simply call „claude -p“ or „codex exec“.

For simple workflows this seems good enough and can be set up in 10 minutes without third party software.

What do you think?

Re: Agents need control flow, not more prompts

#266

Post by throawayonthe »

i gave in and bought a month of claude (it really is a slot machine don't do it if you have an addictive personality lol) to vibecode a bit, and the Superpowers skill set is cool and all but it really seems like something that should be turned into a program

hmmmmmm maybe i could vibecode a harness based on that pi thing i've heard about, and integrate it closer with jj instead of relying on llms knowing how to use it, and make certain stages guaranteed to run... oh dear

edit: also i can't bring myself to believe the 'ultimate' form or whatever stabilizes out will be chat-based interfaces for coding and code generation

i think it's just that openai happened to strike gold with ChatGPT and nobody has time to figure anything else out because they've got to get the bazillion investor dollars with something that happens to kinda work

also afaiu all these instruct models are based on 'base' models that 'just' do text prediction, without replying with a chat format; will we see code generation models that output just code without the chat stuff?

Re: Agents need control flow, not more prompts

#267

Post by cloaky233 »

It's not that agents don't need more prompts, actually breaking the prompt into a dynamically changing prompt and a static prompt combination does resolve most of the issues. Control flow on the other hand is harnessing + context building, which is one major part of agentic workflows. So I believe a "optimized" combination of both is what we should be looking for.

Re: Agents need control flow, not more prompts

#268

Post by Imanari »

sowbug wrote: post #42
bwestergard wrote: post #3

I agree with the sentiment, but I think the conclusion should be altered. When you hit the limit of prompting, you need to move from using LLMs at run time to accomplish a task to using LLMs to write software to accomplish the task. The role of LLMs at run time will generally shrink to helping users choose compliant inputs to a software system that embodies hard business rules.

A full-circle system prompt would be to "find every opportunity to put yourself out of your job by automating it away. When you are given a question that code can answer, answer the question by writing code and running it to obtain the result." Such an LLM might have fared better with the strawberry test.

That’s exactly the approach of smolagemts. The only “tool“ available is writing python code

Re: Agents need control flow, not more prompts

#269

Post by Frost1x »

krzyk wrote: post #258
fny wrote: post #229

Earlier quoted context omitted.

Secret: "compile" that orchestration prompt. Determinism is solved by turning prompts into code that can in turn run agents or run code or both. Everyone misses this pattern with skills: you can just drop code alongside a SKILL.md to guarantee certain behaviors, but for some reason everyone's addicted to writing prompts. You don't even need to build a CLI. A simple skill.py with tasks does it. You can even have helpe…

Could you elaborate what does "compiling orchestration prompt" mean?

When you get some abstraction working you concretize it in something deterministic, or sort of “cache” that knowledge bit (aka write me a function, class, library, whatever). In the future, the nondeterministic path now has a deterministic piece to lean on as it explores the problem space. Rinse, repeat, eventually you have a mostly deterministic system now. Leave flexibility in space where you need that nondeterminism.

Re: Agents need control flow, not more prompts

#270

Post by Neywiny »

gf000 wrote: post #246
Neywiny wrote: post #48

Earlier quoted context omitted.

Because it's not possible . There is nothing you can say to the LLM that will guarantee that something happens. It's not how it works. It will maybe be taken into consideration if you're lucky. But if you're trying to tell me that every time you list criteria you get them all perfectly matched, you're clearly gifted.

I'm being deliberately pedantic, but depending on what kind of representation we use for the neural network (due to rounding) as well as the choice of inference (that is, given a distribution for next token, which one to choose), it can absolutely be reproducible and completely deterministic. Though chaotic, which I believe is the better word here - a single letter change may result in widely different results. We ju…

With determinism you're not wrong. The problem is that you'd need to make sure all your seeds, temperatures, and other input parameters are exactly the same, and importantly that all context is cleared. But people don't do that. And I'm not sure every if even any provider lets you set those parameters.
Post reply on HN