Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: How do you keep track of software requirements and test them?

news.ycombinator.com

Post reply on HN
Print view View original

121–130 of 134 posts

Re: Ask HN: How do you keep track of software requirements and test them?

#121

Post by quartesixte »

karmakaze wrote: post #78
postingposts wrote: post #64

Earlier quoted context omitted.

Waterfall and Agile are tools. If you need to hang a photo, a hammer and a nail. Cut down a tree? Maybe not the hammer and the nail.

Could you use both to good effect? Waterfall to make a plan, schedule, and budget. Then basically disregard all that and execute using Agile and see how you fare. Of course there would be a reckoning as you would end up building the system they want rather than what was spec'd out.

These are all just heuristics that help people manage the fundamentally unmanageable: the unpredictable future. Everyone does a little bit of everything when working. A big company will waterfall year long strategies with the individual parts agile’d. Individuals will waterfall their daily tasks while working on an agile sprint.

Re: Ask HN: How do you keep track of software requirements and test them?

#122

Post by yshrestha »

Been working as a consultant and engineer on FDA regulated software for about 8 years now. I have seen strategies from startups to huge companies.

I have seen requirements captured in markdown files, spreadsheets, ticket management systems like Redmine, Pivotal, Jira, GitLab, Azure Devops, GitHub Issues, and home grown systems.

If I had to start a new medical device from scratch today, I would use Notion + https://github.com/innolitics/rdm to capture user needs, requirements, risks, and test cases. Let me know if there is interest and I can make some Notion templates public. I think the ability to easily edit relations without having to use IDs is nice. And the API makes it possible to dump it all to yaml, version control and generate documentation for e-signature when you need it. Add on top of that an easy place to author documentation, non-software engineer interoperability, discoverable SOPs, granular permissions, and I think you have a winning combination.

yshrestha@innolitics.com

Re: Ask HN: How do you keep track of software requirements and test them?

#123

Post by jtwaleson »

Woberto wrote: post #76
jtwaleson wrote: post #30

This is super interesting and incredibly difficult. In some regulated environments, like medical devices, you MUST keep track of requirements in your product's technical documentation. I work on a Software Medical Device product and have seen tons of workflows at similar companies. There are many different approaches to this and none that I have seen work really well. In my view this field is ripe for disruption and…

For option A, how do you put the requirements in the repo? Another user mentioned the possibility of having a "req" folder at the same level of e.g. "src" and "test". Maybe the file structure would match that of the other directories? And what do you use - excel files, word docs, .md files, something else?

There are some tools like https://doorstop.readthedocs.io/en/latest/ that streamline it.

Re: Ask HN: How do you keep track of software requirements and test them?

#124

Post by [deleted] »

DyslexicAtheist wrote: post #97
airbreather wrote: post #95

Earlier quoted context omitted.

For safety system software most people I know would be very nervous (as in, I'm outta here) about testing software components and then not testing the end result as a whole, just too many possible side effects could come into play, including system wide things that only reveal themselves when the entire program is complete and loaded/running. What you describe already occurs to some extent in the process and machiner…

to reiterate on parents endorsement for agile and the point that you seem to be taking issue with: nothing in Agile says you can't run final acceptance tests or integration tests before shipping. we have done this in quite a couple of companies where things like functional safety or other requirements had to be met. agile sadly gets a bad rep (as does devops) for the way it is rolled out in its grotesque perverted st…

[deleted]

Re: Ask HN: How do you keep track of software requirements and test them?

#125

Post by jzer0cool »

jmyeet wrote: post #100
flyingfences wrote: post #28

In a safety-critical industry, requirements tracking is very important. At my current employer, all of our software has to be developed and verified in accordance with DO-178 [0]. We have a dedicated systems engineering team who develop the system requirements from which we, the software development team, develop the software requirements; we have a dedicated software verification team (separate from the development…

Well… the 737MAX seems to suggest it doesn’t catch all the bugs.

If you haven't seen, there is a Netflix documentary worth watching all the way about the 737 Max.

Re: Ask HN: How do you keep track of software requirements and test them?

#126

Post by airbreather »

arthurcolle wrote: post #103
airbreather wrote: post #87

Earlier quoted context omitted.

Actually most functional safety projects use the v-model (or similar, topography can vary a little as to needs), which is waterfall laid out a slightly different way to more clearly show how verification and validation closes out all the way back to requirements with high degrees of traceabilty. I've always wanted to break that approach for something a little more nimble, probably by use of tools - but I can't see ag…

what does functional safety mean in the context you are talking about? like fighter jets? or what?

LNG Plants, Burner Management Systems, Mine Winders, Conveyors - any process plant or machinery where there is potential for harm to come to humans and the is an electronic programmable device mitigating the risk, eg a Safety PLC running a Safety Instrumented System.

I am about to do some automotive FS, so that is potentially ISO 26262, but it might actually be more 61508, which is the parent standard for the safety group of standards.

Re: Ask HN: How do you keep track of software requirements and test them?

#127

Post by airbreather »

DyslexicAtheist wrote: post #97
airbreather wrote: post #95

Earlier quoted context omitted.

For safety system software most people I know would be very nervous (as in, I'm outta here) about testing software components and then not testing the end result as a whole, just too many possible side effects could come into play, including system wide things that only reveal themselves when the entire program is complete and loaded/running. What you describe already occurs to some extent in the process and machiner…

to reiterate on parents endorsement for agile and the point that you seem to be taking issue with: nothing in Agile says you can't run final acceptance tests or integration tests before shipping. we have done this in quite a couple of companies where things like functional safety or other requirements had to be met. agile sadly gets a bad rep (as does devops) for the way it is rolled out in its grotesque perverted st…

I would very much like to understand how it might be possible to improve on the conventional workflow for the work I am involved in. But I am not quite clear what agile as you implement it means, in contrast to v-model and waterfall, and how it provides advantages (I am guessing accelerated schedule?) to the process.

Can you refer me to any available online case studies etc, or provide me some more detail?

The sectors I work in we integrate off the shelf hardware such instruments, valves etc, we don't manufacture from components as such.

Re: Ask HN: How do you keep track of software requirements and test them?

#128

Post by jcon321 »

lovehatesoft wrote: post #74
jcon321 wrote: post #19

Gitlab. Just use Issues you can do everything with the free tier. (It's called "Issues workflow" - gitlab goes a little overboard though, but I'd look at pictures of peoples issues list to get examples). My opinion would be to not use all the fancy features that automatically tie issues to merge requests, releases, epics, pipelines etc... it's way to much for a small team that is not doing any type of management. Jus…

If possible, could I get your opinion on a specific example? In my current situation, I was asked to add a feature which required a few (java) classes. So - * It seems like this would have been a milestone? * So then maybe a few issues for the different classes or requirements? * For each issue, after/during development I would note what tests are needed, maybe in the comments section of the issue? Maybe in the descr…

I don't know your deployment schedule or rules. I represent milestones as groups of independent issues (bug fixes or new features) that would all go together as a release. I don't use milestones as a group of multiple issues that represent one requirement (that would be referred to as an epic). However epics are part of the paid version, however there's no reason why you couldn't use milestones this way.

If you have a requirement (doesn't matter how big or small) I'd treat that as 1 issue (regardless of how many java classes or lines of codes need modifying). If the issue is complex then within the issue's description you can use markdown (like checkboxes or bullet points) to identify subset requirements. However, if you can break that large requirement into functional changes that could exist/be deployed separately then I'd probably do multiple independent issues with some type of common identifier in the issue's name (or use your interpretation of milestones and put all those issues into 1).

If you use gitlab as your git repository then tying an issue to a merge request is easy and it would then show you the diff (aka all the changes to source code) that the issue required for implementation.

In terms of tests, same kind of answer - I don't know your rules. Every issue should have a test plan, perhaps using markdown in the issues description would convey that test plan the easiest. If you automate the test using junit then not sure the test plan is anything more than "make sure test xyz from junit passes", if it's a manual test then the issue's description can have a list of steps using markdown.

Re: Ask HN: How do you keep track of software requirements and test them?

#129

Post by mike22 »

mtoddsmith wrote: post #117
adrianmsmith wrote: post #16

I think it's important to keep requirements in Git along with the source code. That way when you implement a new feature you can update the requirements and commit it along with the code changes. When the PR is merged, code and requirements both get merged (no chance to forget to update e.g. a Confluence document). Each branch you check out is going to have the requirements that the code in that branch is supposed to…

What happens when your requirements span multiple projects and repositories and microservices including frontend and backend work etc.? That doesn't all fit into a git PR.

In the world I live in (big org using Jira), that sounds like an epic, with lots of stories underneath. Some stories are defined early on, more can be added as the business comes up with new reqs or devs find things that need more work.

Re: Ask HN: How do you keep track of software requirements and test them?

#130

Post by maerF0x0 »

airbreather wrote: post #127
DyslexicAtheist wrote: post #97

Earlier quoted context omitted.

to reiterate on parents endorsement for agile and the point that you seem to be taking issue with: nothing in Agile says you can't run final acceptance tests or integration tests before shipping. we have done this in quite a couple of companies where things like functional safety or other requirements had to be met. agile sadly gets a bad rep (as does devops) for the way it is rolled out in its grotesque perverted st…

I would very much like to understand how it might be possible to improve on the conventional workflow for the work I am involved in. But I am not quite clear what agile as you implement it means, in contrast to v-model and waterfall, and how it provides advantages (I am guessing accelerated schedule?) to the process. Can you refer me to any available online case studies etc, or provide me some more detail? The sector…

I recommend you learn more about SAFe agile https://www.scaledagileframework.com/ . They all have their merits, but I find this works up to complex organizations, and can simply drop the things that are not worth it for smaller businesses.
Post reply on HN