Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why Software Factories Fail (or: harness engineering is not enough)

github.com

Post reply on HN
Print view View original

281–290 of 296 posts

Re: Why Software Factories Fail (or: harness engineering is not enough)

#281

Post by tcoff91 »

sevenseacat wrote: post #222
viccis wrote: post #210

Earlier quoted context omitted.

Like you said, they sucked back then but it was more rare to get PRs of many thousand lines to review multiple times a day in my experience. But I also run into a problem now in which I have coworkers in two camps. There are some that, when they send me a PR and I have serious concerns about it, it's because they disagree with me fundamentally on some concept and we need to chat and discuss it. And then there's some…

I received two 15,000-line PRs the other day, from the same person. I am currently ignoring them.

I very much remember whose PRs I can trust and whose PRs are likely to be slop, and I prioritize my time accordingly.

Slop merchants get slop reviews. I just point my clanker at their PRs and ask them to critically review the PR and they typically find lots of issues.

Re: Why Software Factories Fail (or: harness engineering is not enough)

#282

Post by antonvs »

RugnirViking wrote: post #257
shermantanktop wrote: post #208

Earlier quoted context omitted.

Or fire the PM?

PM = politics maestro They will out-corporate you. Better to be friends with them, or failing that, move from under them

I’ve outlived many PMs and helped get at least one fired. If they were omnipotent they’d be better at product design.

Re: Why Software Factories Fail (or: harness engineering is not enough)

#283

Post by hansvm »

Terretta wrote: post #272
hansvm wrote: post #90

Earlier quoted context omitted.

FWIW, I notice the intelligence drop drastically with long contexts with Opus 4.6. It's barely usable for anything intricate and long. That long window is good for _something_, but it's not as good as a short window.

for synthesis. read the entire whatever in one gulp and boil it down or plan what to do about it. not for steps.

Oh yeah, that's a use case I love long contexts for. "Here's a day of prod logs, what's broken?" It's an incredible utility, but the qualities much less than with shorter contexts, and you see it much more apparently when you do similar coding tasks with progressively larger windows.

Re: Why Software Factories Fail (or: harness engineering is not enough)

#284

Post by throw10920 »

reinitctxoffset wrote: post #159
orsenthil wrote: post #136

I find it amusing that people who are talking about Dark Software Factories, are talking about productivity in terms of number of pull requests or commits as a unit. If we are going in the Dark Software Factory route, why aren't we calling the code units as bos (bunch of shit) yet.

No one serious about the idea can afford either vanity metrics nor ignorance of the code. The bar is higher, not lower, to operate with this much automation in the water supply. It's mostly a lot more math and a lot more work. It's an extreme form of any startup: you trade off capital for years of your life.

Interesting, I'd like to discuss further - can you please add an email to your HN profile?

Re: Why Software Factories Fail (or: harness engineering is not enough)

#285

Post by ffsm8 »

softwaredoug wrote: post #261
ffsm8 wrote: post #160

Earlier quoted context omitted.

> If you built a real factory, you’d basically never want it to be dark This statement seems to be factually untrue as the fully automated factories in China are indeed not illuminated for most of the time In USA/Europe we just don't have that level of automation

The “fully automated” factories have humans involved in QA, maintenance and engineering. They often have to stop the factory and investigate and troubleshoot.

i mean youre most likely not wrong (i dont have insider knowledge of those places), however your previous comment said "never want it to be dark" - when the reality is that its normal operation _is_ dark. only when the automated QA detects issues or theyre doing random inspections are the lights turned up/on

this is an outdated video for waht was state of the art a few years ago https://www.youtube.com/watch?v=MCBdcNA_FsI

again, i dont have insider knowledge so i cannot speak on whats SOTA in chinese automated factories beyond whats covered publicly

Re: Why Software Factories Fail (or: harness engineering is not enough)

#286

Post by danny_codes »

pydry wrote: post #231
weatherlite wrote: post #215

Earlier quoted context omitted.

> Either you need to understand how your codebase works or you don’t. It's an interesting point. We can also think about it perhaps as a non binary thing - you need X amount of understanding in a specific codebase to be effective. Even before LLMs in large codebases no one understood it all; but we at least mostly understood our own PRs and our own areas of expertise in the codebase.

Before LLMs you either built good abstractions to make it possible to not understand large chunks of the code base or you flailed. A lot of the time people flailed. The one thing Ive never seen an LLM do well is shape clean, coherent abstractions. To be fair it's a rare human skill as well but in LLMs if they don't have a direct analog in their training data they flail.

Maybe I'm doing something wrong, but the code I get from LLMs would not be considered acceptable to merge before we collectively stopped having standards. People are committing 900 line functions with sprinkled return statements and it's getting into production. If there are productivity gains to be had I haven't seen them yet

Re: Why Software Factories Fail (or: harness engineering is not enough)

#287

Post by kolesnikov-arch »

gyulai wrote: post #247
romanovcode wrote: post #242

Earlier quoted context omitted.

It's just a long ad for his product - humanlayer.

> NO EVIDENCE AT ALL It's funny how, when the hype is strong enough, the burden of proof around the need for EVIDENCE suddenly shifts. Normally, the burden of proof is on $NEWFANGLED_THING to prove it's better than $TRIED_AND_TESTED. Software dark factories where no one looks at code are that unproven newfangled thing and all he's really saying is that, in his experience/assessment, those don't work, so he's trying t…

[flagged]

Re: Why Software Factories Fail (or: harness engineering is not enough)

#288

Post by iamwil »

eloisius wrote: post #267
iamwil wrote: post #254

I've been building and running my software factory for 8 months now. Granted, there's no automated pulling down tasks and pushing PR right now (soon!). But after specifying what I want, it mostly goes to shipping on its own. On occasion, it does raise issues that I have to make a decision on. After doing systems evals on review, I've stopped looking at code during review for 4 months now. I do spend a lot of time up…

I’m most curious about what you’ve built with the code factory. Is it just building itself or do you have any outputs that validate it for building something other than AI coding tools?

It's not building itself, though I did build a harness. It's a collaborative workspace for humans and AI.

Re: Why Software Factories Fail (or: harness engineering is not enough)

#289

Post by iamwil »

chrisweekly wrote: post #264
iamwil wrote: post #254

I've been building and running my software factory for 8 months now. Granted, there's no automated pulling down tasks and pushing PR right now (soon!). But after specifying what I want, it mostly goes to shipping on its own. On occasion, it does raise issues that I have to make a decision on. After doing systems evals on review, I've stopped looking at code during review for 4 months now. I do spend a lot of time up…

I'd be interested in seeing more of your setup, eg if you published a long blog post and/or repo.

Other people have expressed interest too. Will let you know when I'm done.

Re: Why Software Factories Fail (or: harness engineering is not enough)

#290

Post by ghostinit »

_doctor_love wrote: post #2

> I haven't been able to dig up any definitive data/findings from StrongDM on how that whole dark factory went. The weather-report has a few sparse updates between February and June of this year. This was easy to find out I thought. And just with an old-fashioned google search too, no deep research agent needed. See here: https://diffusion.io/ Seems like it went pretty well if a consulting company is now being starte…

[flagged]
Post reply on HN