Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Missing Covid-19 test data was caused by the ill-thought-out use of Excel

bbc.com

Post reply on HN
Print view View original

71–80 of 834 posts

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#71

Post by hdjdbtbgwjsn »

throwmeaway_pls wrote: post #34
chimprich wrote: post #26

Earlier quoted context omitted.

> It's unlikely this will have had much impact since most reported cases were only a few days late I'm not sure if you're being sarcastic or not, but in case you're not, this has meant c.16k people missed from having their contacts traced. That's time-critical. Delays of any number of days are very bad. People will likely die because of this error. This is safety-critical software. It should be stress tested and have…

The WHO estimated 750 million cumulative infections on Friday. This error is not unbelievable because the system is just a bit of political theater. There’s nothing more we can do against an endemic human coronavirus. It’s time to move on.

Yeah we should give up on controlling polio, measles etc as well. After all polio only kills 0.5% of cases as well.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#72

Post by hdjdbtbgwjsn »

ajsnigrutin wrote: post #59
Mvandenbergh wrote: post #2

It's unlikely this will have had much impact since most reported cases were only a few days late but come on guys, really? Using an Excel sheet as a database? In week 1, that would be "not great" but could be accepted as being a fast solution that everyone could work with. This far into a pandemic I think we can expect a little more professionalism in data handling.

Once you have a couple of thousands of rows in excell files, one file per day, possibly one per testing center.... possibly with inconsistent structure.... non standard date formats (even different in each row),... add to this the inconsistent namings ("Tested at" column with data: "Testing center 1, London", "TC1, London", "TC1 London", "TC1 LONDON",....), etc... Yes, anything-sql or whatever would be better, but de…

Cleaning data is a major part of data engineering there are plenty of professionals capable of helping with this task.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#73

Post by makomk »

The thing that puzzles me is, what on earth happened on the 28th? A really anomalously low number of cases were reported then, and it's only got more weird after this fix added zero unreported new cases on that day and much higher numbers on the days before and after it. Something definitely doesn't seem right here.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#74

Post by RhysU »

CamperBob2 wrote: post #21
underdeserver wrote: post #18

The real, visceral tragedy that is COVID-19 notwithstanding, I think it brings a marvelous example of why statistics, the importance of accuracy in data collection, and the effects of exponential growth are a must-have in today's highschool curriculums.

This. We need to drop calculus, even geometry if we have to, in order to make room in the curriculum for statistics. Statistical illiteracy kills.

Change in deaths over change in statistical illiteracy is positive eh? How positive? Presumably the change in statistical illiteracy depends upon the magnitude of the deaths. It'd be an interesting relationship to study...

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#75

Post by raxxorrax »

glandium wrote: post #19
throw_m239339 wrote: post #14

Earlier quoted context omitted.

I mean let's call a cat a cat: this is cronyism thus corruption. Unfortunately, voters never care about these immoral practices. When a social system/administration is dysfunctional, corruption is often the first cause of these disfunctions. The result of corruption is always sheer incompetence.

OT: are you French? AFAIK, calling a cat a cat is a French idiom. The English equivalent would be calling a spade a spade.

A spade is only a cat if you have tomatoes on your eyes.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#76

Post by [deleted] »

glandium wrote: post #19
throw_m239339 wrote: post #14

Earlier quoted context omitted.

I mean let's call a cat a cat: this is cronyism thus corruption. Unfortunately, voters never care about these immoral practices. When a social system/administration is dysfunctional, corruption is often the first cause of these disfunctions. The result of corruption is always sheer incompetence.

OT: are you French? AFAIK, calling a cat a cat is a French idiom. The English equivalent would be calling a spade a spade.

[deleted]

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#77

Post by 231435359236 »

Mvandenbergh wrote: post #2

It's unlikely this will have had much impact since most reported cases were only a few days late but come on guys, really? Using an Excel sheet as a database? In week 1, that would be "not great" but could be accepted as being a fast solution that everyone could work with. This far into a pandemic I think we can expect a little more professionalism in data handling.

I think it's more interesting on what happens to any analysis done with this incorrect data. There was a news article last week suggesting that the growth in COVID-19 cases was "levelling off" and I wonder in retrospect would this incorrect data account for such a position.

The article in question: https://www.bbc.co.uk/news/health-54387057

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#78

Post by iso1631 »

makomk wrote: post #70
iso1631 wrote: post #35

Earlier quoted context omitted.

Or maybe it wasn't excel at all. One number the UK government doesn't release is how many people are tested. They have spent the summer championing "testing capacity", which we found out at the start of September was a lie. They release the number of tests done, and the number of people returning positive. They don't release the number of people done -- if you're tested twice, that counts as two on the number of test…

The trouble is, that entire narrative about the testing capacity being a lie because of secret leaked figures about the number of people tested was itself a massive lie. For context, the Government publishes a stat for the number of people tested for the first time ever each week (used to be daily, but that was dropped in part due to people misusing it). Certain publications and politicians - starting I think with th…

We know capacity was a lie because people were unable to get tests despite the total number of tests being below claimed capacity.
Post reply on HN