Ask HN: How do you keep track of software requirements and test them?

Anyway, no point on tracking low-quality requirements that end up being redefined as you build the airplane in flight.

If working within a safety-critical industry and wanting to do Agile, typically you'll break down high-level requirements into sw requirements while you are developing, closing/formalizing the requirements just moments before freezing the code and technical file / design documentation.

It's a difficult thing to practice agile in such an industry, because it requires a lot of control over what the team is changing and working on, at all times, but it can be done with great benefits over waterfall as well.

then only bump up to more complexity or superficiality as the benefits exceeds the cost/pain. for example, a spreasheet. perhaps a Google doc

if you're lucky enough to have any reqs/specs which have a natural machine-friendly form, like an assertion that X shall be <= 100ms then go ahead and express that in a structured way then write test code which confirms it, as part of a suite of assertions of all reqs which can be test automated like this

I've always wanted to break that approach for something a little more nimble, probably by use of tools - but I can't see agile working in functional safety without some very specific tools to assist, which I am yet to see formulated and developed for anything at scale. Also, there are key milestones where you really need to have everything resolved before you start next phase, so maybe sprints, dunno.

The thing about doing waterfall/v-model is if done correctly there is little chance you get to the final Pre-Start Safety Review/FSA 3, or whatever you do before introducing the hazard consequences to humans, and a flaw is discovered that kicks you back 6 or 12 months in the design/validation/verification process. This, while everyone else stands around and waits because they are ready and their bits are good to go, and now you are holding them all up. Not a happy day if that occurs.

FS relies on high degree of traceability and testing the software as it will be used (as best possible), in it's entirety.

So not sure how agile could work in this context, or at least past the initial hazard and risk/requirements definition life cycle phases.

FS is one of things where your progress that you can claim is really only as far as your last lagging item in the engineering sequence of events. The standard expects you to close out certain phases before moving onto subsequent ones. In practice it's a lot messier than that unless extreme discipline is maintained.

(To give an idea of how messy it can get in reality, and how you got to try and find ways to meet the traceability expectations, sometimes in retrospect - last FS project I was responsible for design we were 2.5 years in and still waiting for the owner to issue us their safety requirements. We had to run on a guess and progress speculatively. Luckily we were 95%+ correct with our guesses when reconciled against what finally arrived for requirements)

But, normally racing ahead on some items is a little pointless and likely counterproductive, unless just prototyping a proof of concept system/architecture, or similar activity. You just end up repeating work and then you also have extra historical info floating around and there's possibility that some thing that was almost right but no longer current gets sucked into play etc etc etc. Doc control and revision control is always critical.

Background: I am a TUV certified FS Eng, I have designed/delivered multiple safety systems, mainly to IEC 61511 (process) or IEC 62061 (machinery).

I've been writing software professional for over 20 years, in all kinds of different industries. I've had to handle thousands of lines of specs, with entire teams of manual testers trying to check them. Worked at places where all requirements were executable, leading to automated test suites that were easily 10 times bigger than production code. Other places just hope that the existing code was added for a reason, and at best keep old working tickets. And in other places, we've had no tracking whatsoever, and no tests. I can't say that anyone was wrong.

Ultimately all practices are there to make sure that you produce code that fits the purpose. If your code is an API with hundreds of thousands of implementers, which run billions of dollars a month through it, and you have thousands of developers messing with said API, the controls that you'll need to make sure the code fits purpose is going to be completely different than what you are going to need if, say, you are working on an indie video game with 5 people.

Having to long terms requirements tracking can be very valuable too! A big part of documentation, executable or not, is that it has to be kept up to date, and be valuable: It's a pretty bad feeling to have to deal with tens of thousands of lines of code to support a feature nobody is actually using. Reading documentation that is so out of date that you will end up with the completely wrong idea, and lose more time than if you had spent the same time reading a newspaper. Every control, every process, has its costs along with its advantages, and the right tradeoff for you could have absolutely nothing to do with the right tradeoff somewhere else. I've seen plenty of problems over the years precisely because someone with responsibility changes organizations to a place that is very different, and attempts to follow the procedures that made a lot of sense in the other organization, but are just not well fit for their new destination.

So really, if your new small team is using completely different practices than your previous place, which was Enterprise enough to use any IBM Rational product, I would spend quite a bit of time trying to figure out why your team is doing what they do, make sure that other people agree that the problems that you think you are having are the same other people in the team are seeing, and only then start trying to solve them. Because really, even in a small team, the procedures that might make sense for someone offering a public API, vs someone making a mobile application that is trying to gain traction in a market would be completely different.