Nobody has built a software factory

I don’t think it’s possible to build a software factory unless you truly do not care about what you’re putting out.

Agent swarms, self learning, Ralph loops, execution DAGs, spending hours trying to convey my preferences into skills, yada yada — the reality is that no matter how long I spend trying to close the amortization gap between my expertise and (insert autonomous agent loop) … I always come back to see shit that I have opinions about and need to polish.

If you’re keeping on top of your craft, I just don’t think general purpose pretraining / RL is going to give you something which perfectly copies you. It can get closer and closer! But I’ve never seen agent work where I didn’t immediately start thinking of refinements …

I guess many things are okay if you’re okay with averages

Exactly. Or at least software is a custom tooling.

A factory in the traditional sense is a process and infrastructure tuned to run efficiently that produces something of value that does not require highly skilled/expensive labor.

I propose a software farm.

Imagine a large multi dimensional grid. Each axis of the grid represents a spectrum of how a specific feature could be implemented. You thus have one axis per feature.

Now you plot a cloud of random points all over the grid and at each point have an agent build a solution where the features are expressed according to the corresponding intensity values at the point.

Now you go through and review the solutions you like, create more points around clusters where you kind of like what is going on, until the perfect product emerges.

I definitely agree with the sentiment of this article, give or take. I've seen it at the place a previously worked at and now the place I currently work at: everybody wants AI to make coding faster and more efficient, but nobody is willing to take the risk of changing course and bending the firm's processes around what would actually make the agents more useful. Maybe this is more general than the qualms against "software factories," but it's pretty apparent that agentic development doesn't mesh with most of the development processes everyone was following prior to LLMs. Whenever a human is in the loop, it creates THE bottleneck which ultimately sets the pace. When humans can't fathom not being in the driver seat, then they're constantly in the loop, and nothing actually improves.

I DO see small instances where this is overcome, albeit, it occurs when risk is minimal, the work is non-critical, and the required environment is already available. For example, exploratory research of new features. Before, this would be quite costly and particularly cumbersome since the results may indicate that the new feature isn't viable and the work is basically "wasted." Now, you can send some agents to explore it for you and give you a "first pass" on the validity of the concept. You still end up with humans taking over immediately after (often to verify that first pass), but this has allowed a bit more efficiency in terms of understanding when investing that human time in exploratory investigation is appropriate and when it isn't. People are willing to give agents this level of control because the alternative is usually just a default "no."

Regardless, it seems obvious we're in a painful transition period where everyone in software can see that there's a ton of value here, but where everyone is simultaneously constrained by existing infrastructure, processes, expectations, etc., so as to not be willing or able to tap into that value. I suspect it won't be until there's just more new companies coming in without these existing constraints will the actual new version of these processes start to emerge and MAYBE some of the existing companies will be more willing to shake things up as needed to take advantage of it.

> its self-improvement loop ends where everyone’s does: “Humans review those suggestions as PRs on the factory definition and merge improvements."

That's because it's still painfully clear that genAI has no taste. It's a median content generator. And the median kinda sucks. Of course you still need people to review the output.