101–110 of 296 posts
Earlier quoted context omitted.
I think the right shape is to review and merge directly from the agent window.
reviewing code as it's being written & re-steering it > reviewing it once 20k lines have been written
It reminds me of all the recent talk about "taste". Architecture "quality" may not be objective in a right/wrong sense, in the same way that fashion isn't right/wrong. It's like we are all going to have to relinquish reason/rationality to the machine and start to study up on aesthetics.
Even historically, my big struggles have usually been deciding between two nearly-equivalent options. I get this a lot now with LLMs because there is no break in-between these decisions that implementation used to force. I feel I'm constantly making "taste" calls between tradeoffs that have no clear objective criteria, and it is as exhausting as the code review this post (and my experience) suggests are still necessary, even with Fable/GPT-5.6 level models.
In many cases, I do what I've done with junior engineers whose code I reviewed pre-agent: make on-the-spot judgement calls. When I see a broken window, I call it out. But when I see minor issues, I sometimes just let it pass, note it in memory and tackle it wholesale once an accumulation of similar minor issues get to a certain size.
As a tangential aside, I consider two dev shops from pre-agent days. One decides to hire 7 extremely talented engineers and gets them to work closely together. The other decides to outsource to 100 decent engineers and tries to silo them into modules. I think we are facing a similar choice with agents. You can either work extremely closely with a handful of agents, collaborating on design, review, etc. Or you can spin up a fleet of sub-agents and YOLO, then try to separate the wheat from the chaff in some automated way. My taste is the former, small highly coordinated shop. But time will tell if I am right or wrong.
There's some good ideas and points in here, but this bit threw me: > # We tried this > In July 2025 we went full lights-off Isn't it pretty well-accepted at this point that the models underwent a step-change in usefulness around fall 2025 / spring 2026? I know that I was able to start handing agents whole features after that, but not before. I feel like any perspective/experience on "what agents can/can't do" from be…
Did you even read further on? The section right after that addresses the step changes in current frontier models https://github.com/humanlayer/advanced-context-engineering-f...
It is comforting to find other people experiencing the exact same reality as me, since I see so much in this post that matches my own experience. It reminds me of all the recent talk about "taste". Architecture "quality" may not be objective in a right/wrong sense, in the same way that fashion isn't right/wrong. It's like we are all going to have to relinquish reason/rationality to the machine and start to study up o…
"you know a bad pattern when you see it because at some point you were up at 2am debugging it"
taste is the hard-earned intuition about every anti-pattern and landmine that has blown up in your face since you started doing software
I've recently started experimenting with grounding LLM driven implementation/verification on RFC based normative specifications, to avoid having to manually steer the LLM during implementation and dealing with reviewing sloppy pull requests. It works quite well, as it puts your entire focus on writing (hopefully) unambiguous specifications vs. having to discuss unwanted changes with an LLM during code-review. One fla…
Earlier quoted context omitted.
appreciate that context! I definitely did not mean to come out and say "its definitely not working" or anything, but would love to hear from y'all a retrospective on the ~5-6 month anniversary - what was right, what did we get wrong, etc
We are working on new articles to share our latest findings, so stay tuned! Overall our outlook continues to be bullish. Almost all software problems yield to a combination of the Factory Techniques covered on the strongdm.ai website. More powerful models work even better...
It is comforting to find other people experiencing the exact same reality as me, since I see so much in this post that matches my own experience. It reminds me of all the recent talk about "taste". Architecture "quality" may not be objective in a right/wrong sense, in the same way that fashion isn't right/wrong. It's like we are all going to have to relinquish reason/rationality to the machine and start to study up o…
yeah my best articulation of taste is something i got from Jake Nations[1] while he was still at netflix - "you know a bad pattern when you see it because at some point you were up at 2am debugging it" taste is the hard-earned intuition about every anti-pattern and landmine that has blown up in your face since you started doing software 1 - https://www.youtube.com/watch?v=eIoohUmYpGI
The hand-wringing on the 50-50 cases is almost always theoretical, in the sense that I am trying to reason instead of rely on memory/experience. But that is a double edge sword because sometimes memory/experience are closer to trauma and can lead one away from a solution that is correct now in some new environment.
It also reminds me of the phrase "use in anger". You don't really know about an approach/architecture until you've had to deal with something in an urgent or high-stakes moment (your 2am debugging). But to get that experience, you have to have shipped the thing first. It is a chicken-and-egg problem, you can't debug something at 2am unless it is live, and you can't know if it will cause you a problem until you've dealt with it.
AI (LLMs, agents, etc) is this giant alteration to the environment that shakes up everything. In some sense, I feel I have to throw out my taste and "use in anger" all over again. It is painful but may be the only way. And I have to accept the risk that going slowly (like this post suggests) may be the wrong way, and I may watch the young untraumatized new-comers blast by me riding their 100 agent orchestrations to massive success. What I mean here is, in the final scenario from my original comment (7 top guys vs 100 decent guys) - I know the result from experience. But am I just traumatized and it will be different with agents?
To me, the thing that stands out about the whole state we're in here is PR review. Yes, in an ideal world, PRs read well, are a joy to review, reflect what you discussed etc etc. We have to be real; there is only so much we can do to that end. I'm not sure how the best teams do PR review, from my perspective it sucks . I'm talking specifically about the UX. I've always hated Github's PR page, so I typically reviewed…
Gating integration behind code review is futile. I (and many other engineers) already automated it. My agent responds to review requests and reviews as me. Company policies enforcing human code review are futile. I think all these platforms chasing code review are doomed. My LLM doesn't need any of this tooling. We should be reviewing the actual working software. Systems that make it easy and instant to demo any prop…
Doesn’t this gloss over the fact that token subsidization is a thing? I assume the reason a lot of people switched from other harnesses to CC was all the weirdness around setup-token and whether third party harnesses would continue to work without api pricing.
> Claude Code won because of Reinforcement Learning inside the harness Doesn’t this gloss over the fact that token subsidization is a thing? I assume the reason a lot of people switched from other harnesses to CC was all the weirdness around setup-token and whether third party harnesses would continue to work without api pricing.