Live data from Hacker News

Why Software Factories Fail (or: harness engineering is not enough)

github.com

221–230 of 296 posts

Re: Why Software Factories Fail (or: harness engineering is not enough)

#221

Earlier quoted context omitted.

> few users, no revenue or minimal revenue The 900 pound gorilla has for a while been "where is all the output of these LLMs?", and this seems to be the answer; the number of apps on the various app stores has skyrocketed over the past two quarters but installs overall are actually down a little bit. So it seems like we've had an explosion of software that one or two people use. Which is a cool situation to think abo…

Marketing was a problem with apps even before AI, which, with today’s slopforests makes it even harder for consumers to see any of your trees even if they wanted to.

100 points for the term Slopforest :-)

Re: Why Software Factories Fail (or: harness engineering is not enough)

#222
post #210
post #21

Earlier quoted context omitted.

PRs sucked to review long before Agents were a thing, but now it really sucks because there are more to review.

Like you said, they sucked back then but it was more rare to get PRs of many thousand lines to review multiple times a day in my experience. But I also run into a problem now in which I have coworkers in two camps. There are some that, when they send me a PR and I have serious concerns about it, it's because they disagree with me fundamentally on some concept and we need to chat and discuss it. And then there's some…

I received two 15,000-line PRs the other day, from the same person.

I am currently ignoring them.

Re: Why Software Factories Fail (or: harness engineering is not enough)

#223
post #69

Earlier quoted context omitted.

Isn't maintainability mostly about applying basic engineering principles e.g. separation of concerns, single responsibility, dependency inversion, open-closed etc..? If these topics are addressed in the normative representation, and correctly translated by an LLM into code (especially by slicing up the specifications into measurable outcomes to set the intended foundation), then future change e.g. maintainability bec…

> Historically I know that the majority maintenance problems occur from slow continuous evolution of a system that it initially was never designed for. And the only way to address this was continuous system design. yes exactly - this is what I'm advocating for - that you can't skip the system design, and that actually good system design goes down to the typedefs and object graph at the code level, not just mermaid ch…

Hey, I’ve watched some of your talks on YouTube and they have generally felt right and influenced my thinking as to how to navigate software engineering in this era. I appreciate you engaging here!

I read your piece here and again I appreciate and agree with your main points. You have a section here titled “30 minutes of planning saves hours of review” — but I find it hard to believe that you are doing all this pre-work of creating specs (ai-assisted) at various levels, down to the level of function signatures and such, and this whole thing only takes you 30 minutes. Is that what you meant? Can you talk a bit about how long this process actually takes, or if it is really only 30 minutes, how you pull that off?

Re: Why Software Factories Fail (or: harness engineering is not enough)

#224
Honest question: For let’s say a senior developer having to go through all of these steps for the model to implement a feature:

> Product Design > System Architecture > Program Design > Vertical Slices

By the time it takes to go through these steps and agree with the agent, wouldn’t the senior dev not just be able to implement the feature by themselves?

I mean, in general; I know there’s always features with complex logic etc, so the question is just about the “general case”

Re: Why Software Factories Fail (or: harness engineering is not enough)

#225
post #179

Earlier quoted context omitted.

You either need to quit or keep your head down, I can’t tell which.

If you need to actively keep your head down, you need to quit. Doesn’t matter whether ‘the problem’ is the employer, the employee, both, or neither. They’re a bad fit for each other.

> If you need to actively keep your head down, you need to quit.

This really depends on whether they are paying you decently. The number of people I've met who found a nice corner in FAANG, and decided to keep their head down and collect a steady paycheque... One may find one can swallow a lot of engineering pride for a recurring six figures

Re: Why Software Factories Fail (or: harness engineering is not enough)

#226
post #152

Earlier quoted context omitted.

It's even worse: Customer says I want X! One or more of the following can be true: - Customer really wants X - Customer doesnt actually need X at all - Customer wants Y not X but couldnt express himself - Customer B and C do NOT want X at all - Customer is just messing with you for no reason

Why deliver one solution? You're thinking small. If LLMs were fast enough, they could build A,B,C,...,Z in parallel. So that's what will end up happening. The operator will just have to make sure there's no combinatorial explosion of the possible requirements.

Right. Because If I want feature A and it is available in configuration among features A through Z I will always perfectly pick the right letter of the alphabet and I will not at all be irritated by having 26 options that I have no idea about what is the difference between them.

Re: Why Software Factories Fail (or: harness engineering is not enough)

#227

Either you need to understand how your codebase works or you don’t. Claude can write the code for you but it can’t understand it for you. That part has to happen at human speeds. There are cases where you don’t have to understand everything, but I think that’s a more nuanced question. All of the above is true even if Claude writes perfect code.

> Claude can write the code for you but it can’t understand it for you

Part of the problem is that Claude can go spelunking in a spaghetti codebase far faster than us humans. It doesn't necessarily help you understand the project, but if you farm that understanding out to Claude, you can keep working long after the codebase has become unusable for humans...

Re: Why Software Factories Fail (or: harness engineering is not enough)

#228

Earlier quoted context omitted.

I agree this rocks I do this almost daily

It goes hand in hand with Linux desktops too, because everything is just so much more scriptable. That used to be intimating, user unfriendly, and maybe it still is. But with AI it enables so many new avenues.

Now that LLMs can do the actual configuration, I started using NixOS and now all my config lives in one repo, and every quality of life improvement fans out to every machine without issue.

Since Claude (code) knows all about my system, it will also tailor solutions to it when I ask how to do something. Using it I can often find really neat solutions to whatever sys admin task I can imagine.

Re: Why Software Factories Fail (or: harness engineering is not enough)

#229
post #38

My radical opinion is that LLMs are harmful for software development - they are the ultimate "goto" operator. All actual code should be written by a human developer. Instead, use them in adversarial mode - run QA scenarios using LLM agent as a substitute for end user to do bug discovery.

> All actual code should be written by a human developer. This seems arbitrary. Why don’t you say the same thing about machine code? Developers use tools so they can avoid writing machine code. What is causing you to draw a line in the sand about use of tools? The obvious answer is it’s just a function of the time period you grew up in, and a lack of willingness to adapt to change.

This is indeed an important angle to consider as well.

I think there is some nuance though, but it's hard to articulate. I think that is because there are some long term effects that are currently invisible, but that can be anticipated. A lot of it is the human factor, and how changes to how people think and act may ripple through the organizations as well.

Things such as skill atrophy which may reduce not only immediate skills, but also adjacent skills that were maybe important for critical thinking and making good decisions. Or the fact that learning to use an LLM is a skill that is not truly grounded in reality and can therefore not translate well to other areas (you're essentially learning skills within the "reality" of the LLM, based on its weights). That is very different from machine code --> tools, where the tools make explicit assumptions that are based on the workings of the machine code (which is analytical in nature), and where you can, if you wish to, jump in to override those assumptions where needed without having to adapt to a mental model that operates in its own reality (i.e. the LLM's "thought process").

Re: Why Software Factories Fail (or: harness engineering is not enough)

#230

Either you need to understand how your codebase works or you don’t. Claude can write the code for you but it can’t understand it for you. That part has to happen at human speeds. There are cases where you don’t have to understand everything, but I think that’s a more nuanced question. All of the above is true even if Claude writes perfect code.

> Claude can write the code for you but it can’t understand it for you Part of the problem is that Claude can go spelunking in a spaghetti codebase far faster than us humans. It doesn't necessarily help you understand the project, but if you farm that understanding out to Claude, you can keep working long after the codebase has become unusable for humans...

Ive often wondered what people do when the mess is so bad that Claude hits a wall and nothing works.
Post reply on HN