Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Backpressure is all you need

lucasfcosta.com

Post reply on HN
Print view View original

41–50 of 133 posts

Re: Backpressure is all you need

#41

Post by kami23 »

_zoltan_ wrote: post #9

"In this post, I’ll cover a third, not-so-obvious approach: building ways for the agent to validate more of its own work before a human has to step in. " this has been an obvious thing to do since at least January (since Geoffrey Huntley published "everything is a ralph loop"), and this is how I've been working: build enough orchestration tooling to be able to automate everything: development container bringup, build…

This is where most of my productivity gains have come, I have a special harness I move from project to project now that does my testing orchestration, lots of my work day is setting up a prompt or two early and just letting them loop till they return evidence that the feature is working having gone through the big QA loop.

I've slowly been optimizing for token use through the stack and Claude ends up making very tight for loops for most of the process and keeping token count even lower. It's been nice. A lot of my toil at work is just gone.

Re: Backpressure is all you need

#42

Post by entrope »

pshirshov wrote: post #35

A very long post about a simple and very obvious idea with many different implementations. The three main problems are 1) API usage is deadly expensive 2) Claude is about to make all automation very expensive 3) all the flows where a model has the initiative are strictly biased towards unwarranted stops (checkpointing). Also, I won't call that "backpressure", there is no producer-consumer disbalance or something simi…

> all the flows where a model has the initiative are strictly biased towards unwarranted stops

Can you elaborate on what you think causes such a bias? My experience is that Qwen3.6, Claude Sonnet 4.6 and Opus 4.6/4.7 will work as far as they can given direction and a way to test their work. My so-far limited experience with Opus 4.8 is that it does stop somewhat earlier for feedback, but in places where I am glad it is checking assumptions or where I agree with it identifying a change in scope (for example, where the following work deserves a separate commit or merge request). I would call those justified stops rather than unwarranted.

Re: Backpressure is all you need

#43

Post by jon-wood »

This what hooks[1] are for, except hooks allow specifying criteria in certain conditions (like the agent believing it’s done and ready to hand back to the user) in a manner that the agent won’t just forget about once it’s a few turns deep, and doesn’t require triggering a whole other LLM instance to read some plain text instructions while you hope it interprets them correctly.

It absolutely makes sense to have a system in place that allows the code generated by an LLM to be automatically validated but there’s no need to resort to a non-deterministic system for these sort of deterministic pass/fail conditions.

[1] https://code.claude.com/docs/en/hooks

Re: Backpressure is all you need

#44

Post by Yokohiii »

wellpast wrote: post #4

I’m willing to be wrong but this industry-wide emphasis on AI creative/coding workflows seems way over-engineered. Ime successful creative execution looks like micro-iterations where each output informs the next creative move. I can build something incredibly fast from essentially caveman grunt instructions through an LLM harness, iterating as I go. Optimizing for feeding a huge plan to an agent sounds to me like a n…

LLMs are too flaky for high quality code. On tougher problems it's very common for an LLM to contradict itself and run in circles. It simply doesn't know what the right thing is, but on each turn it is super confident to do the right thing.

Maybe I've chosen hardmode to learn C with LLM assistance, plus my pet project turned out to be a bit less trivial then anticipated. But I know that I have to think three times about my choices how to deal with C problems and seeing how a LLM struggles to give reasonable answers is a a huge red flag and forces me to think about it a fourth time.

Doing all this with a fast autonomous workflow with just little user guidance is asking for trouble.

Re: Backpressure is all you need

#45

Post by cadamsdotcom »

Everyone looking into this and other verification should be moving away from long prompts and complex skills, and looking into hooks.

If you put all these checks in your stop hook and your git commit hook, your repo docs can tell your agent that checks will run automatically when it stops work, and it should fix any problems found.

It’s wonderful to reintroduce determinism at the QA end of your process. I find it very calming to know the agent can’t skip or forget to check its work because with hooks the checks are run by the harness.

Re: Backpressure is all you need

#46

Post by pshirshov »

entrope wrote: post #42
pshirshov wrote: post #35

A very long post about a simple and very obvious idea with many different implementations. The three main problems are 1) API usage is deadly expensive 2) Claude is about to make all automation very expensive 3) all the flows where a model has the initiative are strictly biased towards unwarranted stops (checkpointing). Also, I won't call that "backpressure", there is no producer-consumer disbalance or something simi…

> all the flows where a model has the initiative are strictly biased towards unwarranted stops Can you elaborate on what you think causes such a bias? My experience is that Qwen3.6, Claude Sonnet 4.6 and Opus 4.6/4.7 will work as far as they can given direction and a way to test their work. My so-far limited experience with Opus 4.8 is that it does stop somewhat earlier for feedback, but in places where I am glad it…

Ask Claude! It will quote its constitution aka soulfile. It says the constitution instructs it to perform regular checkpointing no matter what.

Re: Backpressure is all you need

#47

Post by bluGill »

0x696C6961 wrote: post #5
wellpast wrote: post #4

I’m willing to be wrong but this industry-wide emphasis on AI creative/coding workflows seems way over-engineered. Ime successful creative execution looks like micro-iterations where each output informs the next creative move. I can build something incredibly fast from essentially caveman grunt instructions through an LLM harness, iterating as I go. Optimizing for feeding a huge plan to an agent sounds to me like a n…

Yeah it's wild watching so many people decide waterfall is great all of a sudden.

Every large project in the coming back to waterfall. While the problems are certainly known and it was ultimately developed as a straw man, everything else ends up working worse. That said, you shouldn't be thinking pure waterfall as it's drawn up as a strawman, but rather a waterfall variation with feedback loops. But in the end, in very, very many cases, you have to know an end date in order to get things done because so many other things depend on you being done at the same time. If something is going to get done sooner you can't use it anyway without all the other pieces.

Re: Backpressure is all you need

#48

Post by manmal »

cadamsdotcom wrote: post #45

Everyone looking into this and other verification should be moving away from long prompts and complex skills, and looking into hooks. If you put all these checks in your stop hook and your git commit hook, your repo docs can tell your agent that checks will run automatically when it stops work, and it should fix any problems found. It’s wonderful to reintroduce determinism at the QA end of your process. I find it ver…

I think pi-subagents (which can form arbitrarily long chains of subagents, with up to 8 in parallel) and Claude Code‘s new workflows feature, are quite convenient abstractions that can be setup quickly.

Re: Backpressure is all you need

#50

Post by bmm6o »

jeffbee wrote: post #24
xg15 wrote: post #19

Isn't this a bit of an incorrect usage of the term "backpressure"? OP quoted the correct definition right at the start: > In systems engineering, backpressure is the mechanism by which a downstream component signals upstream that it can't accept more work (the "downstream component" being the human reviewer in this case) But the measures they propose don't actually do that. They are more like fixed throttle elements…

It is an incorrect use of what was already a flawed metaphor. Pressure is isotropic. Directed pressure makes no sense, like all other fluid analogies in unrelated fields of engineering.

I took the analogy to be about the location of the pressure and not the direction. If you allow pressure to build on the input pipe when you can't accept more, the component that is upstream in the flow is able to observe that and respond. Maybe the difference is I envisioned a series of pipes and not a single one.
Post reply on HN