Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Software Factories Fail (or: harness engineering is not enough)

github.com

Post reply on HN
Print view View original

141–150 of 296 posts

Re: Why Software Factories Fail (or: harness engineering is not enough)

#141

Post by latentsea »

4lx87 wrote: post #18
rglynn wrote: post #8

To me, the thing that stands out about the whole state we're in here is PR review. Yes, in an ideal world, PRs read well, are a joy to review, reflect what you discussed etc etc. We have to be real; there is only so much we can do to that end. I'm not sure how the best teams do PR review, from my perspective it sucks . I'm talking specifically about the UX. I've always hated Github's PR page, so I typically reviewed…

Gating integration behind code review is futile. I (and many other engineers) already automated it. My agent responds to review requests and reviews as me. Company policies enforcing human code review are futile. I think all these platforms chasing code review are doomed. My LLM doesn't need any of this tooling. We should be reviewing the actual working software. Systems that make it easy and instant to demo any prop…

Entropy is not your friend.

Re: Why Software Factories Fail (or: harness engineering is not enough)

#142

Post by dhorthy »

danpalmer wrote: post #137
rglynn wrote: post #8

To me, the thing that stands out about the whole state we're in here is PR review. Yes, in an ideal world, PRs read well, are a joy to review, reflect what you discussed etc etc. We have to be real; there is only so much we can do to that end. I'm not sure how the best teams do PR review, from my perspective it sucks . I'm talking specifically about the UX. I've always hated Github's PR page, so I typically reviewed…

PR review sucks because engineers often don't optimise their code authoring for it. When they do it's a joy. PR review of agent generated code sucks because agents are, in my experience, very poor at authoring for review. This makes sense because the review process is not represented in the resulting code artifact which is what they're trained on. Agents are always producing much larger changes than should be in a si…

exactly. a great pr is a joy to review. we've found some success in agents generating static HTML walkthroughs that order the diffs in something other than GitHub's default alphanumeric ordering, but it can only go so far

Re: Why Software Factories Fail (or: harness engineering is not enough)

#143

Post by majormajor »

4lx87 wrote: post #18
rglynn wrote: post #8

To me, the thing that stands out about the whole state we're in here is PR review. Yes, in an ideal world, PRs read well, are a joy to review, reflect what you discussed etc etc. We have to be real; there is only so much we can do to that end. I'm not sure how the best teams do PR review, from my perspective it sucks . I'm talking specifically about the UX. I've always hated Github's PR page, so I typically reviewed…

Gating integration behind code review is futile. I (and many other engineers) already automated it. My agent responds to review requests and reviews as me. Company policies enforcing human code review are futile. I think all these platforms chasing code review are doomed. My LLM doesn't need any of this tooling. We should be reviewing the actual working software. Systems that make it easy and instant to demo any prop…

> We should be reviewing the actual working software. Systems that make it easy and instant to demo any proposed change are what is needed. Code (and specs) are going to fade into obscurity. PR review has already shifted towards validating the product (working software) over the process (code).

Ok, sure, if your software can be easily exhaustively re-tested on every change.

But in that case, why were you ever doing PR's in the first place? Cargo cult?

Re: Why Software Factories Fail (or: harness engineering is not enough)

#144

Post by mrothroc »

dhorthy wrote: post #39
jadar wrote: post #33

> So, why can't models do software maintainability? I feel like the explanation does nothing to actually elucidate why models can't do it. Is it an inherent weakness of LLMs? Training processes? The typical "this is crap" that we constantly hear? It goes on to write about RL and how there's no penalty for bad design. But that sort of side-steps the question and makes you ask: "why not do RL and make a penalty for bad…

fair point, this is the thing I struggled most to extract out while writing it - if you can propose an RL environment that penalizes a model for bad design, then I'm all ears - right now there's no fast oracle/verifier for this (as stated in the post) My current evolving take on "how would you build such a thing" is you need to tee up a roadmap of 20 features and feed them to a model one at a time, so it can't design…

[flagged]

Re: Why Software Factories Fail (or: harness engineering is not enough)

#145

Post by dhorthy »

orsenthil wrote: post #136

I find it amusing that people who are talking about Dark Software Factories, are talking about productivity in terms of number of pull requests or commits as a unit. If we are going in the Dark Software Factory route, why aren't we calling the code units as bos (bunch of shit) yet.

its the optimizing utilization instead of overall throughput all over again. eli goldratt talked about this in the 1970s[1]. we still haven't learned

1 - https://en.wikipedia.org/wiki/The_Goal_(novel)

Re: Why Software Factories Fail (or: harness engineering is not enough)

#147

Post by janalsncm »

Either you need to understand how your codebase works or you don’t.

Claude can write the code for you but it can’t understand it for you. That part has to happen at human speeds.

There are cases where you don’t have to understand everything, but I think that’s a more nuanced question.

All of the above is true even if Claude writes perfect code.

Re: Why Software Factories Fail (or: harness engineering is not enough)

#149

Post by sathish316 »

I call it the Intent-Implement-Quality problem.

Software factories can implement anything given a one-liner requirement. That one-liner requirement can be a complete app/product, epic, feature, bug, design change or refactoring. But these one liner requirements are requirements coming from a human who has an intent or requirement or direction for the product to evolve in mind. Can Software factories manufacture intent that reflects the exact requirements of the person using it or their vision of how the product or software should evolve?

Implementation in any language is as easy as generating a summary for an LLM. If coding is only math and there is only one way to translate a requirement into an implementation, the problem boils down to just providing or verifying generated intent. But, there is definitely more than one way to implement a thing and one of the ways leads to a design and architecture that is coherent with rest of the system, design and architecture that is extensible and evolves, code that you can keep in your head when you want to change things next time or tomorrow, software that can serve millions of users and securely guarantee millions of dollars of revenue. There is a combinatorial explosion of ways and the one right way is also subjective of the person and the problem. Software factories can of course improve Quality by generating more unit tests, more integration tests, fix code violations, generate Proof of work as videos, screenshots etc, but there is no feedback loop or test suite to verify and correct the other subjective notions of Quality.

Post reply on HN