Live data from Hacker News

To software engineers criticizing Neil Ferguson’s epidemics simulation code

blog.khinsen.net

111–120 of 189 posts

Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code

#111
post #74
post #52

Earlier quoted context omitted.

> If you compare the free support academic software developers receive from the rest of their community to other engineering disciplines, it's beyond great. Could you elaborate on what you mean by that?

Not the OP, but I know of at least one University that runs "Programming for scientists" courses at low or no cost.

Oh we teach programming alright, but software development is another pair of shoes entirely. Unfortunately, that's where the problem lies...

Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code

#112

Earlier quoted context omitted.

The quickest decision is random number generation. Should people go with that? Governments across the world had loads and loads of time. > Adding research capacity to ensure that governments have a stable of well-researched, thoroughly-vetted models for emergencies would be a great thing, but it would also be quite expensive. Keeping any single model up to spec might be the job of 1FTE (so an additional ~$150k/grant/…

>If you have a once in a century crisis, and still are thinking about saving some millions, while being sure your crisis response will be above billions, your policy is already flawed and not much can be done to help you. By the time you're in the crisis, it's too late to make software bulletproof. Not only does that process take time, but it also needs to be integrated with the whole of the research effort up to tha…

But nobody is asking for bullet proof software here. Nobody is asking perfect MC/DC coverage.

If governments are so incompetent that they can't forsee a crisis like this even one month in advance, where even sanitizer hoarders have better foresight, then off course nothing can really help. Governments do spend billions and trillions of money on national defence, and an issue like this when it's almost at the door should be given the zame priority as national defence.

Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code

#113

This letter feels as though it is overlooking a large point of contention. >The scientists who wrote this horrible code most probably had no training in software engineering, and no funding to hire software engineers Shouldn't the argument be, that for research that is reliant on coding models, funding be allocated to experts that can assist in creating said models (software engineers)?

For this to happen there would need to be a nationally accepted PE certification for 'software engineers' on par with other engineering disciplines. It's unreasonable to expect non-experts, and experts from other fields, to perform credentialing on a case by case basis.

Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code

#114
I just took a quick skim through the repository and I'm not quite sure what everyone is so upset about. It looks like simulation code.

I would argue that the real problem isn't C++ but tooling that is aimed at producing source code as an artifact as opposed to repeatable executions as artifacts. There are effectively lots of models in this system, and the code represents all of them tangled together. But that means that you have source code, parameters, tracing, and output as a thing.

Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code

#115

I'm trying to be charitable on this code base issue, and the institutions will need to accept they need to be able to bear more scrutiny on the inputs to policy recommendations. Both R and Python are taught in highschool and undergraduate courses for scientific data analysis. Ferguson and other scientists did not need software engineers. A free co-op student, or to have spent the three weeks learning a language that…

I honestly don't think R and Python would have worked here. While I was able to clone the repo and run it for Ireland, the UK requires at least 26Gb of RAM, which is not common on most personal computers. The US requires much, much more. And given that it's pretty slow when written in C++, imagine how slow it would have been in R or Python? I agree with you in principle with respect to this stuff being better, but th…

Any serious attempt at modelling this over python would use the pydata stack (numpy, pandas, etc), which run on top of C++ anyways.

Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code

#116

Earlier quoted context omitted.

I'm not sure I would describe R as "easier" than C++ tbh.

It's much more accessible, as you need to know less to use it. Consider - using Eigen in a C++ project, or calling install.packages("eigen") in R.

It's easier to dip your toes into, but it is a terrible language to learn software development with. It's such a hodge-podge of ill-thought-out and ill-fitting components with random names and no overall sense of structure that you're never going to learn core CS concepts from it, let alone good programming practice.

Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code

#117
post #78

Earlier quoted context omitted.

If you're willing to dismiss all companies as optimizing for more money, it seems only fair to say that academics optimize for prestige and publication in good journals.

That's also true. Now the million $ question: Whom would you like our society to rely on to generate quality work? I don't think there is a satisfactory answer to this question. Public research becomes more and more of an industry every year with the publish-or-perish game, while a solely private solution is obviously open to very biased conclusions. There is no smart solution to a stupid problem. But the truth is th…

In most fields, our society relies on private industry to generate quality work, even when the work is very important and doing it wrong might kill people. I'm not an anarchist, I do recognize there are reasons that the government should provide some things. But the idea that private industry uniformly produces bad results because they don't care about anything but profit just seems silly to me. Producing good results is profitable!

Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code

#118
post #111
post #74

Earlier quoted context omitted.

Not the OP, but I know of at least one University that runs "Programming for scientists" courses at low or no cost.

Oh we teach programming alright, but software development is another pair of shoes entirely. Unfortunately, that's where the problem lies...

That's the whole problem with the "Learn to code" movement, tbh. Plus Python and Java are terrible languages when it comes to those broader swdev/softeng concerns.

Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code

#119
post #86

John Carmack has reviewed the code and didn't seem to find it all that bad, so there's that. https://twitter.com/ID_AA_Carmack/status/1258192134752145412 https://twitter.com/ID_AA_Carmack/status/1244302925855326209

That's a valuable pointer, thanks for sharing :-)

Re: To software engineers criticizing Neil Ferguson’s epidemics simulation code

#120
post #33

I poked at the github repo for a bit. The ugliness of the code doesn't bother me, but the quantity of parameters does. Here's one params file that specifies some of the inputs to a run of the model: https://github.com/mrc-ide/covid-sim/blob/master/data/param_... Here's another one: https://github.com/mrc-ide/covid-sim/blob/master/data/admin_... There are hundreds of constants in there. A lot of them appear to be wild…

I was asked to look at the spatiotemporal parameters and modeling, separate from any code issues. That part of the model is astonishingly naive, apparently oblivious to existing research and science on the matter that strongly recommends a different and much more nuanced approach. Industry has invested inordinate amounts of money in understanding how to build effective real-world predictive models of this type and none of that knowledge is reflected here. That seems like a rather glaring oversight and alone voids any utility as a predictive model.
Post reply on HN