To software engineers criticizing Neil Ferguson’s epidemics simulation code
1–10 of 189 posts
Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code
#2The author makes it sound like a warning label is missing from the C++ tin. Maybe. But what tool should he have used? I haven't seen this code but is there any doubt it would look just as bad in Java or Python, maybe with fewer segfaults? Or FORTRAN.
Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code
#3I get what he’s saying, but the “non-experts” using code to represent their models should also let outside help come when the need arises.
When your model predicts an apocalyptical scenario and government is taking drastic measures based on it, it’s a good time to expose your “non-expert code” to the software engineering communities (and all other associated fields) to take a look.
Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code
#4The author makes it sound like a warning label is missing from the C++ tin. Maybe. But what tool should he have used? I haven't seen this code but is there any doubt it would look just as bad in Java or Python, maybe with fewer segfaults? Or FORTRAN.
I'm genuinely surprised it wasn't an Excel spreadsheet.
Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code
#5I get what he’s saying, but the “non-experts” using code to represent their models should also let outside help come when the need arises. When your model predicts an apocalyptical scenario and government is taking drastic measures based on it, it’s a good time to expose your “non-expert code” to the software engineering communities (and all other associated fields) to take a look.
This is starting to happen through Research Software Engineers, although as it is still a (relatively) new movement within academia.
Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code
#6Not a comment on the specific repo in question, but I just want to note that I have seen utter monstrosities of academic code written in Python, MATLAB, and R - languages that are ostensibly "easier" than C++. I so not think that poor code quality is due to the many footguns C++ admittedly gives you.
I am sure that the main research is not in the implemented code. But with unclear code, it is exceedingly hard to know that there are no mistakes: that the researched models have been properly encoded.
That, I believe, is what software engineers are afraid of.
Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code
#7this is quite off the mark. the author sets the bar too low for himself by criticizing the most easily (and to be fair legitimately) dismissed criticisms of the Imperial College model.
here's a better laid-out critique that the OP doesn't speak to:
The Imperial College modelers released the source code a couple of days ago to the model that shut down the world economy. It's not the original model code but was rather original source code turned over to volunteer programmers who re-wrote it so that is more readable. I have done some model review of financial models in the past but without the source code I would not be able to do a full review of the Imperial College model. Now that we have the source code (sort of), I can.
Any such model ought to have been independently reviewed before it is ever used for real policy decisions. Policy analysis is awash in models but no one ever really checks them. Going forward, health policy makers should ask for and disclose independent validation of any model before using its results to make recommendations of any consequence.
Normally, model reviews are long technical documents but there would also be a summary section. Here's what I think a summary should have looked like.
Overall conclusion: this model cannot be relied on to guide coronavirus policy. Even if the documentation, coding, and testing problems were fixed, the model logic is fatally flawed, which is evidenced by its poor forecasting performance.
https://www.facebook.com/scarlett.strong.1/posts/25243721950...
Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code
#8> Consider what you, as a client, expect from engineers in other domains. You expect cars to be safe to use by anyone with a driver’s license. You expect household appliances to be safe to use for anyone after a cursory glance at the instruction manuals. It is reasonable then to expect your clients to become proficient in your work just to be able to use your products responsibly? Worse, is it reasonable to make that expectation tacitly?
I don’t buy this argument. Cars are safe, yes. C++ isn’t the car, though. C++ is the dangerous machine shop you use to build the car. A better analogy would be comparing a car to a web browser, and indeed incredible effort has been put into keeping web browsers secure.
Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code
#9The scientific method is supposed to be reproducible. Others should be able to follow the steps and arrive at the same outcome. Scientsits working in wetlabs are not publishing papers on what they do in a language only they can understand and expecting others to blindly trust the results and conclusions.
Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code
#10I get what he’s saying, but the “non-experts” using code to represent their models should also let outside help come when the need arises. When your model predicts an apocalyptical scenario and government is taking drastic measures based on it, it’s a good time to expose your “non-expert code” to the software engineering communities (and all other associated fields) to take a look.
I don't think the government is taking drastic measures based solely on this code. It's just a way to model facts we already know. We know that viruses spread exponentially, and we know what exponential growth looks like this just lets use model different assumptions to see how they affect that model.