Absent those skills, the agile movement is flailing with GenAI.
Why Software Factories Fail (or: harness engineering is not enough)
171–180 of 296 posts
Re: Why Software Factories Fail (or: harness engineering is not enough)
#172Either you need to understand how your codebase works or you don’t. Claude can write the code for you but it can’t understand it for you. That part has to happen at human speeds. There are cases where you don’t have to understand everything, but I think that’s a more nuanced question. All of the above is true even if Claude writes perfect code.
Re: Why Software Factories Fail (or: harness engineering is not enough)
#173To me, the thing that stands out about the whole state we're in here is PR review. Yes, in an ideal world, PRs read well, are a joy to review, reflect what you discussed etc etc. We have to be real; there is only so much we can do to that end. I'm not sure how the best teams do PR review, from my perspective it sucks . I'm talking specifically about the UX. I've always hated Github's PR page, so I typically reviewed…
PRs sucked to review long before Agents were a thing, but now it really sucks because there are more to review.
Re: Why Software Factories Fail (or: harness engineering is not enough)
#174Earlier quoted context omitted.
>I (and many other engineers) already automated it. My agent responds to review requests and reviews as me. Company policies enforcing human code review are futile. This is also known as being a terrible engineer. If a company enforces human review and someone deliberately tries to circumvent this with an LLM, I'd fire that person in an instant The reason for human code review is: 1. So *you* understand what's going…
There are also compliance reasons. SOX requires code review by a second party. If the reviews are automated, the development process isn't in compliance and that can get expensive in a hurry.
Re: Why Software Factories Fail (or: harness engineering is not enough)
#175I find it amusing that people who are talking about Dark Software Factories, are talking about productivity in terms of number of pull requests or commits as a unit. If we are going in the Dark Software Factory route, why aren't we calling the code units as bos (bunch of shit) yet.
Re: Why Software Factories Fail (or: harness engineering is not enough)
#176Earlier quoted context omitted.
GPT 5.6 Sol and Fable are similar step changes, not 5.5, but that's beside the point. If in the Markdown instruction files you ask them to periodically clean up the code or refactor where necessary then that's what they'll do. They don't explicitly do it because otherwise people, as they had for previous models, complain where models are asked to change one thing then they rewrite whole systems, so now the latest mod…
i guess to clarify my contention: generic prompting like "review this code" or "make the architecture better" will raise the floor but cannot come close to human-quality code without humans understanding what code exists and what to ask for. Obviously cursor and other labs have a stake in this going one direction, but I don't buy it, and I don't think you should either. In fact, "Rebuild sqlite from spec" has all the…
Re: Why Software Factories Fail (or: harness engineering is not enough)
#177I think there is a fundamental issue here of what building software even means If you think you can just assign Github tickets to AI agents and go drink daiquiris on the beach I think you'll find that you end up with more and more towers of abstraction and indirection. There are 'points of view' that emerge during coding I think. And at some point you as a human have to be like "wait... what if we use Redis here". "W…
My experience has been that current models are way beyond all but the most highly competent developers I've encountered in my several decades career.
Re: Why Software Factories Fail (or: harness engineering is not enough)
#178There's some good ideas and points in here, but this bit threw me: > # We tried this > In July 2025 we went full lights-off Isn't it pretty well-accepted at this point that the models underwent a step-change in usefulness around fall 2025 / spring 2026? I know that I was able to start handing agents whole features after that, but not before. I feel like any perspective/experience on "what agents can/can't do" from be…
I had a bit of this impression when reading the post as well as the authors' product website. A lot of it does seem to be stuck in 2025. For instance I think their post "long-context isn't the answer" on their website post straight-up isn't accurate, and gives me the impression they are just extrapolating previous performance to new models. In my experience, Opus 4.6 and newer have worked very reliably for long conte…
I have countless examples of this for Fable 5 and Opus 4.8 at just half that many tokens. Sometimes they can't even remember the correct workflow even if they're merely an orchestrator and have been going through the same 5 step loop multiple times for pretty much their entire context window.
Wall of shame with an Opus and a Fable oopsie from just yesterday: https://i.imgur.com/znSJ3OE.png and https://i.imgur.com/DLOHQ1B.png
One of them is particularly egregious because it uses a CLI tool to step through the stages that literally tells it what to do next. How do you fuck this up? Needless to say that tool was revised to force the agents to acknowledge everything happened, but I'm sure it will find a way to eventually mess up anyhow.
And orchestration should be the easy part. It's pretty much just writing and adjusting workflow files. Don't get me started on code quality - but I realize that's subjective and my bar is unusually high. There's a reason I have them run adversarial verifiers along multiple axis before their output is even worth my attention.
Re: Why Software Factories Fail (or: harness engineering is not enough)
#179Earlier quoted context omitted.
> If you’re working on an app or software that has few users, no revenue or minimal revenue, tolerance to bugs is higher and just another Claude prompt away, Software factories are a perfect fit. Most personal software or hobby software or 0-1 yet-to-find-PMF startups belong in this category. I would rather claim that for my personal/hobby software, I apply a much higher quality standard than for the software that I…
You either need to quit or keep your head down, I can’t tell which.
Doesn’t matter whether ‘the problem’ is the employer, the employee, both, or neither. They’re a bad fit for each other.
Re: Why Software Factories Fail (or: harness engineering is not enough)
#180Earlier quoted context omitted.
Agents are also controllable to a degree. How agents author PRs is now an engineering problem.
But this is my point, they're not really controllable in this way. There are no fixed rules (like tests or linters) for small PRs, it's not good enough to do it by line count. A good PR tells a story to the reader that leaves them confident in the result. That means knowing the reader and how they think, and that is something LLMs are very bad at.