Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Missing Covid-19 test data was caused by the ill-thought-out use of Excel

bbc.com

Post reply on HN
Print view View original

211–220 of 834 posts

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#211

Post by dang »

makomk wrote: post #184
dang wrote: post #100

Earlier quoted context omitted.

You're right. But this article contains significant new information (SNI) about Excel, and spreadsheet topics are perennially interesting to HN readers, so this is the rare case where a quick follow-up article, and also the title change, probably makes sense. Usually it goes the other way of course: https://hn.algolia.com/?query=follow-up%20by%3Adang&dateRang... https://hn.algolia.com/?query=%22significant%20new%20in…

The BBC has expanded on the Excel-related part and broken it out into a seperate article: https://www.bbc.co.uk/news/technology-54423988 Maybe it'd make more sense to link that one, if you're keeping the title?

Ok, changed to that from https://www.bbc.com/news/uk-54422505. Thanks!

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#212

Post by remus »

diarrhea wrote: post #210

Each test result creating several rows of data seems like a problem too. In clean data, every observation is one row. It makes working with the dataset much easier. In this scenario, I would expect one observation to correspond to one test result. The multiple rows are then better off pivoted into columns.

Looking from an outside perspective I would agree, but in practice we don't actually know what this dataset is, so several things rows per test may make sense in the context of this stage of the data processing.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#213

Post by dang »

codeulike wrote: post #129

The BBC updated the article with technical details, see about halfway down. Here's the tech details: (PHE = Public Health England) Analysis by Leo Kelion, Technology desk editor ----------------------------------------------- The BBC has confirmed the missing Covid-19 test data was caused by the ill-thought-out use of Microsoft's Excel software. Furthermore, PHE was to blame, rather than a third-party contractor. The…

OK, we'll change to that from https://www.bbc.com/news/uk-54422505. Thanks!

More explanation here: https://news.ycombinator.com/item?id=24690286

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#214

Post by Pick-A-Hill2019 »

Fun semi-related story. Way back when, while I was squishing some Y2K bugs for a Saudi bank that catered to high net-worth individuals I was asked to look into why the AS400==>>db==>>Excel spreadsheet was "showing something screwy every now and then".

Spoiler Alert: It was a SMALLINT error that was 'patched' by someone previously (legacy code) that worked around the bug by rounding account balances to the Million (since it was a reporting function rather than an accounting function it was a 'sort of ok' hack). The bug re-appeared as undefined behaviour when transaction and or account balances went in to the billions.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#215

Post by [deleted] »

supercanuck wrote: post #198
beagle3 wrote: post #85

This is indeed user incompetence, but so far every news source quoted the official PR and said it was an "Excel Problem". Microsoft PR was caught unprepared - I wonder how they'll re-spin it in the next few days (and for the first time that I can recall, a Microsoft product was wrongly blamed...) Pay attention, how every time there's a Windows virus or worm, it's a "Computer Virus", but in the (extremely rare) occasi…

Could also be a lack of budget funding. The newer version of excel supports: 1,048,576 rows

[deleted]

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#216

Post by 1123581321 »

hinkley wrote: post #207

I hate solving problems and not knowing how I did it, so this particular experience has stuck in my craw. Trying to develop a budget to pay off debts, my partner made this elaborate Excel spreadsheet and the output was that basically she had no spending money and I had very little, until one or both of us got a raise. It was far more austere than either of us were willing to go. So I started over using a different eq…

That rework based on suspicion is typically how it’s done. :) Add to that subject matter expert review. For example, I used to work on a sheet of scoop size calculations for industrial digging and nothing from it would reach sales staff or customers until a couple of the crusty old experts played with it and vouched for the calculations.

Imperfect measures, but they keep the work within the grasp of everyday/business users in a way a formal test suite wouldn’t necessarily.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#217

Post by piannucci »

jsty wrote: post #197
mjw1007 wrote: post #181

So, what we know is: « PHE had set up an automatic process to pull this data together into Excel templates [...] When [the row limit] was reached, further cases were simply left off. » The terrible thing here is dropping data rather than reporting an error and refusing to run. It isn't clear what piece of software was behind this "automatic process". Clearly the responsible humans are to blame. If the software that d…

> Is there anything in Excel itself that behaves like that? I'd imagine the default error-handling behaviour of 9/10 Excel macros is to throw away data

Oh, my God. "On Error Resume Next" is one of my earliest memories of learning to program for Windows. It never occurred to me that it might result in deaths.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#218

Post by LandR »

supercanuck wrote: post #198
beagle3 wrote: post #85

This is indeed user incompetence, but so far every news source quoted the official PR and said it was an "Excel Problem". Microsoft PR was caught unprepared - I wonder how they'll re-spin it in the next few days (and for the first time that I can recall, a Microsoft product was wrongly blamed...) Pay attention, how every time there's a Windows virus or worm, it's a "Computer Virus", but in the (extremely rare) occasi…

Could also be a lack of budget funding. The newer version of excel supports: 1,048,576 rows

The test and trace system for the UK was ~£12bn.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#220

Post by philjackson »

codeulike wrote: post #129

The BBC updated the article with technical details, see about halfway down. Here's the tech details: (PHE = Public Health England) Analysis by Leo Kelion, Technology desk editor ----------------------------------------------- The BBC has confirmed the missing Covid-19 test data was caused by the ill-thought-out use of Microsoft's Excel software. Furthermore, PHE was to blame, rather than a third-party contractor. The…

Everyone's saying not checking for errors was the terrible thing, but I think having Excel as a middle-man in such a sensitive system was the first issue.

It's such a British government thing to do too (I worked for Syntegra when I got out of uni).

Post reply on HN