Requirements volatility is the core problem of software engineering • Hacker News Board

Re: Requirements volatility is the core problem of software engineering

#161

Post by moron4hire » Sat, Feb 22, 2020, 5:56 PM UTC

clarry wrote: post #87

bmn__ wrote: post #86

Earlier quoted context omitted.

> instead of sitting down and figuring out the actual requirements first like any real engineering job If it were so easy, then we would have done it already that way for the last seven decades. The simile topples over upon further contemplation: civil engineers: • client is not a domain expert • to an overwhelming part, client needs are easy to transport into the mind of c.eng. • can employ a wealth of standard solu…

I agree it's not easy, but instead of trying harder to become good at it, it seems like this industry in its proverbial baby shoes threw a tantrum and decided that it doesn't want to learn at all. So instead of pushing for more effort & skill for up-front planning and modelling of the requirements and design (and trying to employ modelling tools & formal methods), we're just like fuckit nah let's just implement somet…

We tried all that. Formal requirements gathering, modeling tools, the works. It was the SDLC/Six Sigma/ISO9000/vendor-certs-are-substitutes-for-degrees era of the late-90s/early-2000s. And it also didn't work.

Often times, it's very easy to determine the functional requirements: "Process this billing statement", "Spit out that report." The complexity doesn't come from the requirements. It comes from the constraints: "We only use this ancient version of this database", "We didn't tell you at the start that the users expect to interface with this system via email".

To quote Mike Tyson, "everyone has a plan, until they get punched in the face." The point of Agile is to get punched in the face earlier and more frequently.

Re: Requirements volatility is the core problem of software engineering

#162

Post by moron4hire » Sat, Feb 22, 2020, 6:01 PM UTC

clarry wrote: post #147

mewpmewp2 wrote: post #116

Earlier quoted context omitted.

Didn't we start by trying to plan ahead? There's a reason we have moved to agile. In software if you know what you are doing ahead of time you are not really innovating and you might as well be using an existing system. Since everything that is standardised you can just use as libraries. When building bridges, you can't just use a library that's .buildBridge(...parameters), even though similar things have been done t…

> Didn't we start by trying to plan ahead? There's a reason we have moved to agile. "Everything was waterfall and then came agile" is myth, but yes, it looks like it's fashionable to no longer try to plan ahead. I'm not convinced the reasons are good, and I'm not convinced the craft of planning ahead was ever perfected to the point that anyone could say it doesn't work. Scrum, XP, Agile all date back to the 90s when…

"Not trying" is not new. The problems with both Waterfall and Agile is that nobody really ever did them the way they were intended. Waterfall was supposed to include prototyping during requirements gathering, specifically to aid in discovering constraints. Agile was supposed to give equal footing to all team members and not let the project manager spring new constraints on the team without planning for the change. Neither of those things happened very frequently, because our modern business culture treats knowledge workers as interchangeable minions, and management as unassailably perfect.

Yes, people don't do their jobs. That's not the fault of the process that they are failing to adhere to. Indeed, no process of any type could ever save them. It's the culture that is wrong.

So, in a roundabout way, I agree that Agile isn't the solution. No bottom-up process will ever succeed, because at the end of the day, the failure of the team is always the fault of the leader.

Re: Requirements volatility is the core problem of software engineering

#163

Post by derefr » Sat, Feb 22, 2020, 6:02 PM UTC

rossdavidh wrote: post #108

I am reminded of how linguists, when trying to document a language from one of the last living native speakers, don't just ask them to tell them the rules of it. They have to task questions like "how would you say this?" Then they try to reverse-engineer the grammar, syntax, etc. In most cases, the people who are giving the requirements either: 1) don't actually know them (e.g. they are in upper management but the so…

Reminds me of this recent post on the Grammarphobia blog: https://www.grammarphobia.com/blog/2020/02/adjectives-comple...

> We wish we could tell you that there’s a predictable pattern here—that certain types of adjectives can always be complemented by both participles and infinitives, while other types are always restricted to one or the other.

> Unfortunately, no clear pattern emerges. Different adjectives simply act differently in different contexts.

In other words, sometimes there isn’t a rule, even after intensive analysis. There’s just a bunch of individual one-off decisions that each happen to work to solve their problem, without representing a coherent policy.

Re: Requirements volatility is the core problem of software engineering

#164

Post by moron4hire » Sat, Feb 22, 2020, 6:14 PM UTC

mewpmewp2 wrote: post #116

clarry wrote: post #87

Earlier quoted context omitted.

I agree it's not easy, but instead of trying harder to become good at it, it seems like this industry in its proverbial baby shoes threw a tantrum and decided that it doesn't want to learn at all. So instead of pushing for more effort & skill for up-front planning and modelling of the requirements and design (and trying to employ modelling tools & formal methods), we're just like fuckit nah let's just implement somet…

Didn't we start by trying to plan ahead? There's a reason we have moved to agile. In software if you know what you are doing ahead of time you are not really innovating and you might as well be using an existing system. Since everything that is standardised you can just use as libraries. When building bridges, you can't just use a library that's .buildBridge(...parameters), even though similar things have been done t…

There is a reason we moved to Agile, yes, but it was the wrong solution to the right problem. Scrum identified the problem as a lack of accountability for management, and then imposed new processes on employees without any teeth to hold management accountable. That's why I hate Scrum's too-cutesy-by-half zoomorphization of the problem with the analogy of the chickens and pigs (https://en.wikipedia.org/wiki/The_Chicken_and_the_Pig). In a real barnyard, the pigs and chickens are on equal footing: they are both slaves to the farmer. But in most software dev projects, the developers are not on equal footing with the project manager.

Re: Requirements volatility is the core problem of software engineering

#165

Post by austincheney » Sat, Feb 22, 2020, 6:14 PM UTC

jayd16 wrote: post #155

austincheney wrote: post #121

Earlier quoted context omitted.

As a web developer with 20 years experience I would argue that one web SPA project is more similar to the next than comparing two bridges and developers still can’t figure it out. One difference is that a license is required to practice civil engineering. Most web developers, on the other hand, require large tools to do theirs jobs for them and recoil in irrational fear and disgust when standard DOM methods are used.…

>Most web developers, on the other hand, require large tools to do theirs jobs for them and recoil in irrational fear and disgust when standard DOM methods are used I would recoil in disgust if I saw a civil engineer building a suspension bridge by hand as well.

I am not sure what you mean. When are you not writing code by hand or are you reliant on somebody to write it for you?

Re: Requirements volatility is the core problem of software engineering

#166

Post by qznc » Sat, Feb 22, 2020, 6:19 PM UTC

wpietri wrote: post #104

clarry wrote: post #72

Earlier quoted context omitted.

> I often hear people say that software engineering is a joke compared to civil engineering ... I think your example points out exactly why some think software engineering is a joke: they rush ass first to implement something (and they implement the wrong thing) instead of sitting down and figuring out the actual requirements first like any real engineering job would have you do. And it looks like everyone is pushing…

> figuring out the actual requirements first like any real engineering job would have you do There are no "actual" requirements for most software projects. Software is generally co-evolved with the audience. Bridge designers can work like they do because bridges are hard to change and fundamentally pretty similar. But because software infinitely changeable and infinitely copyable, our field is different. If you're go…

There are plenty of embedded systems which hit 1.0 and stayed there for years and decades.

Re: Requirements volatility is the core problem of software engineering

#167

Post by jayd16 » Sat, Feb 22, 2020, 6:22 PM UTC

moron4hire wrote: post #156

jayd16 wrote: post #155

Earlier quoted context omitted.

>Most web developers, on the other hand, require large tools to do theirs jobs for them and recoil in irrational fear and disgust when standard DOM methods are used I would recoil in disgust if I saw a civil engineer building a suspension bridge by hand as well.

Most software projects are nowhere near the scope of a suspension bridge, and the DOM is certainly a lot higher level than "building by hand". Your analogy is terrible.

Hyperbolic perhaps but the idea that civil engineers don't or shouldn't use tools, tooling, or abstraction is bizarre.

Re: Requirements volatility is the core problem of software engineering

#168

Post by mrdoops » Sat, Feb 22, 2020, 6:23 PM UTC

WhompingWindows wrote: post #157

mrdoops wrote: post #148

This, along with the cost of development, is why I have a pet-peeve for hard-coded business logic directly in the code (in most cases). I.e. when a frequently changing business process like a finite state machine for an approval procedure is only change-able by a developer that means there's always the cost of that developer's "need-to-know" for a straight forward change in the business operations. The cost of a give…

I'm not the most experienced, can you highlight why it'd be wrong to do the following. Let's say you use hard-coded business logic, but you clearly document the code and it'd be clear to any experienced programmer what's going on. How is that un-ideal? Yes it's non-transferrable code, but it's so much faster to write purpose-built code, right?

If the code is clear and it's documented that's probably the best option in most circumstances. The hurdles for making a tenable configuration / logic-as-data system are much higher. Expert/rule-based systems are hard, but usually necessary in most systems of a size.

The question to think about is "how much will this cost the business to change over time?" A developer hour could cost between $30 - 150 / hr depending. For most businesses cost of development completely dwarfs infrastructure/server costs. If a few small changes like accepting options or avoiding hard-coded values takes a tiny bit more time upfront it's worth considering. If you don't know how the procedure will change, best make it clear, concise, and documented so you, or another developer can make it more flexible down the line. This cost and risk of development is also why the "buy first, develop second" strategy tends to be prevalent in IT decision making.

I guess my real pet peeve is that developers tend to make tooling for other developers and don't always put the same effort into non-developer tooling. If you think about the CRUD apps we build- so often it's on a mountain of tooling and abstractions to get it as productive as it is (Database + ORM + Web Framework). We should be focusing that same kind of effort in tooling development at declarative/point-and-click use cases so developer need-to-know out of the change equation more often.

Re: Requirements volatility is the core problem of software engineering

#169

Post by qznc » Sat, Feb 22, 2020, 6:27 PM UTC

markus_zhang wrote: post #92

clarry wrote: post #72

Earlier quoted context omitted.

> I often hear people say that software engineering is a joke compared to civil engineering ... I think your example points out exactly why some think software engineering is a joke: they rush ass first to implement something (and they implement the wrong thing) instead of sitting down and figuring out the actual requirements first like any real engineering job would have you do. And it looks like everyone is pushing…

Manager and Director are the people who are pushing "agile". They want quick results, and SE can only do what they want. You simply really can't "gather" requirement and finish a feature and do proper testing in a 2-week sprint. I do agree there are a lot of bad practices in SE as well, but let's not blame everything on them.

My understanding of Scrum is that requirement elicitation must happen before the sprint. Otherwise the story does not meet the Definition of Ready and the team should reject it.

https://www.scruminc.com/definition-of-ready/

Re: Requirements volatility is the core problem of software engineering

#170

Post by mrdoops » Sat, Feb 22, 2020, 6:27 PM UTC

crgwbr wrote: post #150

mrdoops wrote: post #148

This, along with the cost of development, is why I have a pet-peeve for hard-coded business logic directly in the code (in most cases). I.e. when a frequently changing business process like a finite state machine for an approval procedure is only change-able by a developer that means there's always the cost of that developer's "need-to-know" for a straight forward change in the business operations. The cost of a give…

> This, along with the cost of development, is why I have a pet-peeve for hard-coded business logic directly in the code (in most cases). I’ve built systems both ways. Requiring a developer to make changes definitely has costs, but it does have benefits as well. Systems configured in data rather than code will almost invariably be untested and prone to configuration bugs. Overall, I’ve spent probably double or triple…

Agreed. I guess my point is that we need to focus more on making declarative business rules systems more tenable as an approach in general. If we consider the framework + CRUD generators as the most productive way to build a web app (probably has been since Rails) - that's a result of tooling built around an approach. What happens when we focus similar efforts of tooling towards these Rule Based components?