Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Agents need control flow, not more prompts

bsuh.bearblog.dev

Post reply on HN
Print view View original

241–250 of 348 posts

Re: Agents need control flow, not more prompts

#241

Post by stingraycharles »

est wrote: post #239

I have a question, does LLM follow these MANDATORY or DO NOT SKIP during pre-train, like how people write a comment paragraph on reddit corpus, or is it just some post-train alignment habbit?

Instruction following is a specific fine tuning / post training phase, yes.

That’s why you see “base” vs “instruct” models for example — base is just that, the basic language model that models language, but doesn’t follow instructions yet.

Especially the open weights models have lots of variants, eg tuned for math, tuned for code, tuned for deep thinking, etc.

But it’s definitely a post train thing, usually done by generating synthetic data using other models.

Re: Agents need control flow, not more prompts

#242

Post by dns_snek »

swores wrote: post #119
mmis1000 wrote: post #112

Earlier quoted context omitted.

> This started breaking down after ~30 files. Codex's short context and todolist system combined somehow helps here though. Because of the frequent compact. The model was forced to recheck what todo list item has not done yet and what workflow skill it has to use. I used to left it for multi hour to do a big clean up and it finished without obvious issues.

Is Codex willing to do "multi hour" tasks when used with a ChatGPT Plus subscription, or does it need something more expensive like Pro?

It's going to work the same regardless of how much you pay, but with Plus you'll run into 5h usage limit rather quickly unless your "multi hour task" spends 90% of the time just waiting around for code to compile. Expect to get an hour or two of active work (single-threaded).

Re: Agents need control flow, not more prompts

#243

Post by imtringued »

827a wrote: post #100

1000% agree. I am increasingly hesitant to believe Anthropic's continual war drum of "build for the capabilities of future models, they'll get better". We've got a QA agent that needs to run through, say, 200 markdown files of requirements in a browser session. Its a cool system that has really helped improve our team's efficiency. For the longest time we tried everything to get a prompt like the following working: "…

I'm personally surprised by this too. Like, everyone is writing how insanely productive AI is making them, but that productivity doesn't seem to have translated into any innovations beyond model quality.

Like, most of the stuff needed to make AI better is stuff that could have been written by hand in 2015, so why hasn't anyone used their agents to do so?

To be fair, there is probably a way to make it work the way you want. You could add an MCP for a task queue and let the model work each item in the task queue. The tasks could be added by a deterministic system i.e. your harness.

Re: Agents need control flow, not more prompts

#244

Post by otikik »

827a wrote: post #100

1000% agree. I am increasingly hesitant to believe Anthropic's continual war drum of "build for the capabilities of future models, they'll get better". We've got a QA agent that needs to run through, say, 200 markdown files of requirements in a browser session. Its a cool system that has really helped improve our team's efficiency. For the longest time we tried everything to get a prompt like the following working: "…

I never tell claude to "go over this bunch of files and do this".

I tell it "write a program that goes over this bunch of files and do this".

Sometimes "do this" can be invoking another claude instance.

Re: Agents need control flow, not more prompts

#245

Post by gf000 »

dnautics wrote: post #58
isityettime wrote: post #56

Afaict all harnesses are wrong in this respect, some of them deeply so. Slash commands, for instance, are a misfeature. I should never have to wait for the chatbot finish a turn so that I can check on the status of my context window or how much money I've spent this session. Control should be orthogonal to the chat loop. Even things that have nothing to do with controlling the text generator's input and output are en…

> Slash commands, for instance, are a misfeature. I should never have to wait for the chatbot finish a turn so that I can check on the status of my context window or how much money I've spent this session. Control should be orthogonal to the chat loop. I get what you're trying to say but in practice architecting what you propose is considerably more difficult. Why not build it and try to get hired by one of the bigco…

In what way would it be more complicated? This is pretty basic concurrent programming, we routinely have much much more complex concurrent designs..

Hell, a telegram bot can handle that just fine.

Re: Agents need control flow, not more prompts

#246

Post by gf000 »

Neywiny wrote: post #48
tekne wrote: post #12

Earlier quoted context omitted.

Wait... why? Making an unreliable, nondeterministic system give reliable results for a bounded task with well-understood parameters is... like half of engineering, no? There's a huge difference between "generate this code here's a vague feature description" and "here's a list of criteria, assign this input to one of these buckets" -- the latter is obviously subject to prompt engineering, hallucination, etc -- but so…

Because it's not possible . There is nothing you can say to the LLM that will guarantee that something happens. It's not how it works. It will maybe be taken into consideration if you're lucky. But if you're trying to tell me that every time you list criteria you get them all perfectly matched, you're clearly gifted.

I'm being deliberately pedantic, but depending on what kind of representation we use for the neural network (due to rounding) as well as the choice of inference (that is, given a distribution for next token, which one to choose), it can absolutely be reproducible and completely deterministic.

Though chaotic, which I believe is the better word here - a single letter change may result in widely different results.

We just choose to use more random inference rules, because they have better results.

Re: Agents need control flow, not more prompts

#248

Post by philipp-gayret »

sidcool wrote: post #247

How does one achieve this?

Native integrations with agents, i.e. Claude Code's system of Hooks.

Harnesses, which kick off agents with what to do.

Tools, which show an agent where in a process it is, and what the next step should be.

In my experience I find Hooks to be extremely powerful cross-project. CLI Tools are easy to make also, and work really well for guiding agents.

Re: Agents need control flow, not more prompts

#249

Post by Weryj »

Pure agentic loops with markdown documents as a program 'agentic workflow' is incredible for experimentation, developing and testing your workflow idea.

The second it works, bake the workflow into the harness. Yesterday I was doing just that, and the whole agent loop disappeared because the process could've been condensed into a one-shot request (+1 MorphLLM fast apply) from careful context construction. (It was an Autoresearcher)

Re: Agents need control flow, not more prompts

#250

Post by andy12_ »

827a wrote: post #100

1000% agree. I am increasingly hesitant to believe Anthropic's continual war drum of "build for the capabilities of future models, they'll get better". We've got a QA agent that needs to run through, say, 200 markdown files of requirements in a browser session. Its a cool system that has really helped improve our team's efficiency. For the longest time we tried everything to get a prompt like the following working: "…

Isn't this already possible to implement with skills and subagents? Like have a skill saying "to test these files run this script that executes a subagent for every markdown file, then check the results".
Post reply on HN