It's hard to design and create software for subjects you don't know much about. It's not really a surprise that people create software for their own lives. A lot of open source is focused on programming languages, libraries and things to help you code. It's what most open source people know, and thus what they create. I wonder if it would have been better for CS to have been only offered as a minor or double-major to…
Domain knowledge is important; however, even more important is domain experience. With domain experience, you learn where the bottlenecks and pain points are in your workflows. During my time as an interest rate risk analyst, I had the displeasure of working with a horrible piece of software. It crashed at the most inopportune times, usually near the end of a run. Considering that one run could be on the order of a f…
What this comes down to is safety/reliability vs speed and efficiency. If you can get a workable extraction, even if we are talking GB's of data (never happened to me, we are always below 1GB and we struggle with it because all has to go in Excel), you could work out a good solution. However, most companies use software that makes extracting data impossible (welcome to SAP). Partly this is due to complexity in users rights, partly is just people ignorance.
This is a very complex domain problem, especially because a SaaS most of the times is not an option because we are talking about very sensible data.