Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Imperial College London have released their Covid-19 epidemic simulation

github.com

Post reply on HN
Print view View original

41–50 of 82 posts

Re: Imperial College London have released their Covid-19 epidemic simulation

#41

Post by Gibbon1 »

JauntyHatAngle wrote: post #23
te_chris wrote: post #16

Earlier quoted context omitted.

That issue is unhelpful. Yes, no good tests, but how about suggesting ways it can actually be improved? Why are devs such bores when it comes to things like this? They have released the model, review it and suggest improvements! Don't grandstand "We, the undersigned etc etc" as if that's going to help improve the codebase in the slightest. Carmack is OK with it, he's put his name to reviewing it. That's not to excuse…

I fear that this area is too divisive currently to allow for normal discourse. The downvoting in these kinds of topics have been atrocious on hacker News the last few weeks.

HN is one of the increasing fewer places online where covid19 deniers can congregate.

Re: Imperial College London have released their Covid-19 epidemic simulation

#42

Post by notkaiho »

mshook wrote: post #36
tasogare wrote: post #21

Earlier quoted context omitted.

Because when you have only a very limited number of arguments it’s easier to do it that way than selecting a lib, adding it to the project, reading the doc, trying it and integrating it for real. I add the issue in C#, and after trying 2 or 3 libs (including one by Microsoft) I just gave up.

I hear you but in their case they have more than 130 LOC to parse ARGV. Doing all that manually hurts both code readability and maintainability.

Having seen a discussion about this on a different forum, someone who used to work in academia mentioned that repeatability and consistency are key to academic code, which means often readability and convention may get sacrificed - it works for the author and if someone wants to repeat the experiment it should get the same results without wangling around with many external packages or modules that may change or become obsolete.

Still hurts to read though :P

Re: Imperial College London have released their Covid-19 epidemic simulation

#43

Post by masonic »

lbeltrame wrote: post #38
cataphract wrote: post #31

Earlier quoted context omitted.

I've seen a lot of code coming out of academia that exhibits a similar complete lack of structure and completely ignores the most basic software development best practices (such as having tests). There are exceptions, and research software engineers are a thing, but unsurprisingly most academics focus their careers on the science rather than the code.

I once had to debug a crash in a very popular program used in genomics (for those in the know, it was an early version of "samtools" IIRC) and I found it really hard to read. (In the end the crash was due to corrupted input data, so I fixed the data and stopped debugging)

  the crash was due to corrupted input data
Bad data shouldn't cause a crash. That's just sloppy.

Re: Imperial College London have released their Covid-19 epidemic simulation

#44

Post by unyttigfjelltol »

bencollier49 wrote: post #29
devit wrote: post #27

Is this thing actually being used for policy decisions? Given the complexity, poor language/framework choice (should have used either Rust or Tensorflow), bad code design, and unclear determination of the parameters (especially the fact they don't seem to be estimated from real data with Bayesian inference, or if they are they didn't release that), as well as the ludicrous CPU and RAM usage making it impossible for m…

It's 13+ years old. There was no Rust or Tensorflow when it was originally produced.

13+ years ago the same academic modeler predicted 200 million deaths from a bird flu outbreak that killed 282 people in total.[1]. That's a pretty enormous error-- alarmist publicity-seeking behavior then and now, but the damage was felt more acutely now. If this modeler really wanted a competent simulation it would have been better constructed. That it was not well-constructed is another datapoint indicating how egregious the error was on the part of the goverment in failing to properly vet it.

[1].https://www.express.co.uk/comment/columnists/frederick-forsy...

Re: Imperial College London have released their Covid-19 epidemic simulation

#45

Post by spatulon »

abainbridge wrote: post #7

This code is great fun. Start here to explore the horror: https://github.com/mrc-ide/covid-sim/blob/master/src/CovidSi... I like the way InitModel() crashes (I think) if a global pointer called bmh (short for bitmap header) isn't first initialized by calling InitBMHead() from Bitmap.cpp. I guess it's obvious to academics with giant brains that InitModel() depends on a bitmap existing. But it gets worse - the pointer…

I was part of the GitHub team that helped get the code ready for public release. We fixed a few bugs, reduced memory consumption, made it portable across operating systems, etc., but the code you see is largely what was written by Neil Ferguson and his team.

Given your concerns, I would like to mention two things:

1) the code was originally a single source file, so I'm not surprised the module boundaries are imperfect - the code is many years old and the module boundaries have existed for only a few weeks.

2) this was not written by professional software engineers, but epidemiologists, and they were working with limited budget, time, and programming experience.

Re: Imperial College London have released their Covid-19 epidemic simulation

#46

Post by lbeltrame »

masonic wrote: post #43
lbeltrame wrote: post #38

Earlier quoted context omitted.

I once had to debug a crash in a very popular program used in genomics (for those in the know, it was an early version of "samtools" IIRC) and I found it really hard to read. (In the end the crash was due to corrupted input data, so I fixed the data and stopped debugging)

the crash was due to corrupted input data Bad data shouldn't cause a crash . That's just sloppy.

Indeed: in fact someone must have found the issue eventually, as newer versions exited with an error instead of core dumping.

Sadly, academic constraints (time; results required the day before) prevented me from actually filing a bug report.

Re: Imperial College London have released their Covid-19 epidemic simulation

#47

Post by tasogare »

notkaiho wrote: post #42
mshook wrote: post #36

Earlier quoted context omitted.

I hear you but in their case they have more than 130 LOC to parse ARGV. Doing all that manually hurts both code readability and maintainability.

Having seen a discussion about this on a different forum, someone who used to work in academia mentioned that repeatability and consistency are key to academic code, which means often readability and convention may get sacrificed - it works for the author and if someone wants to repeat the experiment it should get the same results without wangling around with many external packages or modules that may change or becom…

Yep. Also, speaking from academic point of view, code won’t make you advance on your career by itself (only papers based on it will), so the minimum is done until the thing is usable for its goal. So forget about CI/CD, proper test suits, documentation (I hate this point personally and document my projects a bit more than the average researcher) and engineering best practices.

Re: Imperial College London have released their Covid-19 epidemic simulation

#48

Post by zimpenfish »

unyttigfjelltol wrote: post #44
bencollier49 wrote: post #29

Earlier quoted context omitted.

It's 13+ years old. There was no Rust or Tensorflow when it was originally produced.

13+ years ago the same academic modeler predicted 200 million deaths from a bird flu outbreak that killed 282 people in total.[1]. That's a pretty enormous error-- alarmist publicity-seeking behavior then and now, but the damage was felt more acutely now. If this modeler really wanted a competent simulation it would have been better constructed. That it was not well-constructed is another datapoint indicating how egr…

That's an amazingly disingenuous comment.

https://www.theguardian.com/world/2005/sep/30/birdflu.jamess...

He said that if it was the same as the 1918 flu, you could probably scale it up to 200M people. He didn't predict 200M, he speculated that it could be that bad.

And he wasn't alone - "A global influenza pandemic is imminent and will kill up to 150 million people" said "David Nabarro, one of the most senior public health experts at the World Health Organisation"

"A Department of Health contingency plan states anywhere that there could be between 21,500 and 709,000 deaths in Britain."

An unnamed WHO spokeswoman said "best case scenario" would be 7.4 million deaths globally.

https://www.newscientist.com/article/dn7787-flu-pandemic-let...

"And yet, the models show, if targeted action is taken within a critical three-week window, an outbreak could be limited to fewer than 100 individuals within two months."

Oh, he also showed how to react and keep the number of casualties to a minimum.

Don't believe what you read in dirt rags.

Re: Imperial College London have released their Covid-19 epidemic simulation

#49

Post by Gibbon1 »

gpderetta wrote: post #9
notkaiho wrote: post #4

I'm no expert in C++ so not sure if it's meant to look like that, but that code is making my brain hurt.

In my very limited experience, that's what a lot of code coming from academia looks like.

That code reminds me of a program that emulated an old Wyse Color Terminal.

Re: Imperial College London have released their Covid-19 epidemic simulation

#50

Post by SiempreViernes »

lbeltrame wrote: post #10

Some people have raised issues with the model[1][2], or rather, the software implementation of it. There's also a (flagged) submission on HN discussing this[3] referencing [1]. [1] (warning: possibly partisan link) https://lockdownsceptics.org/code-review-of-fergusons-model/ [2] https://github.com/mrc-ide/covid-sim/issues/165 [3] https://news.ycombinator.com/item?id=23099212

> On a personal level, I’d go further and suggest that all academic epidemiology be defunded.

Wow, next they'll review one doctors handwriting and conclude that hospitals should be defunded, with their job handled by horse doctors...

Non-intended randomness is of course bad, but it's bad mainly because it makes it harder to track down causes of actually important problems with the produced distributions.

The worst problem this all out murder attempt can muster is that the code is hard to debug which frankly is should be the default assumption for all research code, not too persuasive. Models are after all just tools: what is critical is that you've made reliable predictions, not that the tools themselves are easy to use correctly.

A more interesting critique would be something along the lines of this: https://www.nicholaslewis.org/imperial-college-uk-covid-19-n... (however it's not by a subject matter expert so the problems they find might well be because the misunderstood some detail).

Post reply on HN