21–30 of 834 posts
The real, visceral tragedy that is COVID-19 notwithstanding, I think it brings a marvelous example of why statistics, the importance of accuracy in data collection, and the effects of exponential growth are a must-have in today's highschool curriculums.
It's unlikely this will have had much impact since most reported cases were only a few days late but come on guys, really? Using an Excel sheet as a database? In week 1, that would be "not great" but could be accepted as being a fast solution that everyone could work with. This far into a pandemic I think we can expect a little more professionalism in data handling.
How do you know it was Excel? I see no technical details in the posted article except "some files containing positive test results exceeded the maximum file size".
Earlier quoted context omitted.
To be fair, there hasn't been any massive irretrievable data loss yet, so not too bad. https://www.youtube.com/watch?v=K_FrQnQv0Vw (probably an accurate depiction of what's going on in Whitehall right now, though)
At this point I think I'd rather have Peter Capaldi acting as Malcolm Tucker running Whitehall over the shower of incompetent people we have now. At least he might get something done in between the flaying
Earlier quoted context omitted.
People use excel sheets as databases all the time because it doesn't require any specialist knowledge to build or use. Sure, we techy types prefer a proper database with a web frontend and an API, but that requires significantly more skill to build than an excel file. No database has the flexibility and flat learning curve that excel has. Until a database manages to meet these goals, it remains a good tool for certai…
> People use excel sheets as databases all the time because it doesn't require any specialist knowledge to build or use. Test and Trace is a multi billion pound programme of work, involving huge companies (eg, Serco) interfacing with national governmental bodies (Public Health England). We've had months to get something in place. One of the reasons organisations like Serco is used is their apparent expertise with IT.
(my one-step programme for improving UK public contracting would be that after a significant failed contract the contracting company and its directors and any other companies they are directors of would be banned from tendering for a period; maybe a couple of years?)
This reminds me of the below event which you can read the full story at https://leveragethoughts.substack.com/p/making-investment-de...
In 1976, the UK government, led by James Callaghan of the Labour, borrowed the sum of $3.9 billion from the International Monetary Fund. The granting of the loan was based on the condition that government fiscal deficit be slashed as a percentage of GDP.
A couple of years later, the chancellor of the exchequer at the time of the IMF loan said this below.
‘If we had had the right figures, we would never have needed to go for the loan”
That’s right!! The decision to borrow money from the IMF was based on wrong data. The public borrowing figures which prompted the UK government to seek IMF loan in 1976 were subsequently revised downwards sometime in the future.
It's unlikely this will have had much impact since most reported cases were only a few days late but come on guys, really? Using an Excel sheet as a database? In week 1, that would be "not great" but could be accepted as being a fast solution that everyone could work with. This far into a pandemic I think we can expect a little more professionalism in data handling.
I'm not sure if you're being sarcastic or not, but in case you're not, this has meant c.16k people missed from having their contacts traced. That's time-critical. Delays of any number of days are very bad. People will likely die because of this error.
This is safety-critical software. It should be stress tested and have sanity checks. It should be impossible for this to happen.
For something like this system, part of a track and trace system costing billions, to fail because of a file size error is just unbelievable.
It's unlikely this will have had much impact since most reported cases were only a few days late but come on guys, really? Using an Excel sheet as a database? In week 1, that would be "not great" but could be accepted as being a fast solution that everyone could work with. This far into a pandemic I think we can expect a little more professionalism in data handling.
> Using an Excel sheet as a database How do you know it was Excel? I see no technical details in the posted article except "some files containing positive test results exceeded the maximum file size".
> The reason was apparently that the database is managed in Excel and the number of columns had reached the maximum.
We did a bit of work for laboratories this year and csv is not an uncommon exchange format between labs. In general almost all exchange formats are text based, with labs saying they will upgrade to „modern xml formats“ at some point in the future. So seen in this context a csv or an excel file doesn‘t really surprise me and should probably also be seen in this context.
I only say that because, as someone who is painfully aware of the limitations and problems of those formats, I'm similarly aware of getting "that web-guy" on a project who proclaims "lets put things in a modern xlm format!", and lo and behold the process is now an order of magnitude slower and the xml format an order of magnitude larger than the simple delimited tabular format or stream.
I'm also painfully aware of the old systems (and how old health systems are) with fixed sized buffers and processes, so I can see how this would happen in the context of a lot of computing.
Edit: i see later on someone is mentioning that twitter suggests it had to do with excel file size limitations...
We did a bit of work for laboratories this year and csv is not an uncommon exchange format between labs. In general almost all exchange formats are text based, with labs saying they will upgrade to „modern xml formats“ at some point in the future. So seen in this context a csv or an excel file doesn‘t really surprise me and should probably also be seen in this context.