Why Software Factories Fail (or: harness engineering is not enough)

Post by dhorthy » Thu, Jul 23, 2026, 6:46 PM UTC

20k wrote: post #22

4lx87 wrote: post #18

Earlier quoted context omitted.

Gating integration behind code review is futile. I (and many other engineers) already automated it. My agent responds to review requests and reviews as me. Company policies enforcing human code review are futile. I think all these platforms chasing code review are doomed. My LLM doesn't need any of this tooling. We should be reviewing the actual working software. Systems that make it easy and instant to demo any prop…

>I (and many other engineers) already automated it. My agent responds to review requests and reviews as me. Company policies enforcing human code review are futile. This is also known as being a terrible engineer. If a company enforces human review and someone deliberately tries to circumvent this with an LLM, I'd fire that person in an instant The reason for human code review is: 1. So *you* understand what's going…

Post by dhorthy » Thu, Jul 23, 2026, 6:47 PM UTC

vanuatu wrote: post #3

This is one of the best writeups I've seen of this a lot of the model's constraints come down to how they are RLed. Discussions online would be a lot better if everyone understood how the labs train the models in a high level (or did a lil data labeling)

Post by 4lx87 » Thu, Jul 23, 2026, 6:48 PM UTC

tripleee wrote: post #20

4lx87 wrote: post #18

Earlier quoted context omitted.

Gating integration behind code review is futile. I (and many other engineers) already automated it. My agent responds to review requests and reviews as me. Company policies enforcing human code review are futile. I think all these platforms chasing code review are doomed. My LLM doesn't need any of this tooling. We should be reviewing the actual working software. Systems that make it easy and instant to demo any prop…

What's your job as an "engineer" in this post-automated world? QA?

Post by [deleted] » Thu, Jul 23, 2026, 6:56 PM UTC

tcoff91 wrote: post #19

fishtoaster wrote: post #15

There's some good ideas and points in here, but this bit threw me: > # We tried this > In July 2025 we went full lights-off Isn't it pretty well-accepted at this point that the models underwent a step-change in usefulness around fall 2025 / spring 2026? I know that I was able to start handing agents whole features after that, but not before. I feel like any perspective/experience on "what agents can/can't do" from be…

Opus 4.5 was a massive step change in capability for sure. The opinions of anyone who hasn't bothered experimenting with these things post-Opus-4.5 are not worth a damn. They still are far from perfect but they're massively helpful.

Post by mrbnprck » Thu, Jul 23, 2026, 6:57 PM UTC

Lindby wrote: post #37

mrbnprck wrote: post #25

I've recently started experimenting with grounding LLM driven implementation/verification on RFC based normative specifications, to avoid having to manually steer the LLM during implementation and dealing with reviewing sloppy pull requests. It works quite well, as it puts your entire focus on writing (hopefully) unambiguous specifications vs. having to discuss unwanted changes with an LLM during code-review. One fla…

But do you actually gain anything if you need to write detailed specifications? That seems just as time consuming as writing code, but less gratifying. Code is just detailed specifications on how things should operate.