The Epistemology of Software Quality

There was no epistemology in this article.

> Static typing? One study, presented at FSE 2014, found no evidence that static typing is helpful—or harmful

And yet the abstract of the linked paper says:

> Most notably, it does appear that strong typing is modestly better than weak typing, and among functional languages, static typing is also somewhat better than dynamic typing.

If you click through the link in that sentence to https://danluu.com/empirical-pl/ or read the study itself, you'll see that the paper doesn't support the claims made in the abstract at all.

It used automatic classification that's obviously wrong. Table 1 gives a list of "top" projects for each language and many of them are simply misclassified.

> ... the "top three" TypeScript projects are bitcoin, litecoin, and qBittorrent). These are C++ projects. So the intermediate result appears to not be that TypeScript is reliable, but that projects mis-identified as TypeScript are reliable. Those projects are reliable because Qt translation files are identified as TypeScript and it turns out that, per line of code, giant dumps of config files from another project don't cause a lot of bugs. It's like saying that a project has few bugs per line of code because it has a giant README. This is the most blatant classification error, but it's far from the only one.

> For example, of what they call the "top three" perl projects, one is showdown, a javascript project, and one is rails-dev-box, a shell script and a vagrant file used to launch a Rails dev environment. Without knowing anything about the latter project, one might expect it's not a perl project from its name, rails-dev-box, which correctly indicates that it's a rails related project.

There are other major problems with the study, but that one is sufficient to make the results invalid.

This article is nice to read, but I believe that some of the arguments are quite flawed. I doubt that empirical evidence is helpful in this context.

Some references to studies are used to make bold statements. The ones on static vs dynamic typing for example, may look scientific, but scanning GitHub repositories or comparing student assignments does not convince me much about professional applications. It just seems nearly impossible to say something evidence based about this topic. Perhaps rewriting the entire Linux kernel along with all its drivers in Rust might be a nice approach. If it were done in parallel, starting in 1991.

Also, applying results from studies in one domain to another is a red flag to me: if productivity drops for construction crews working extra hours, I fail to see how one can conclude that the same holds for software developers. Of course the conclusion is likely, but I don't need a scientific paper for that. The study was from 1980, mind you.

Too bad, because the premise is really interesting and thought provoking. Focusing on the human factor in software development makes a lot of sense.