Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Missing Covid-19 test data was caused by the ill-thought-out use of Excel

bbc.com

Post reply on HN
Print view View original

611–620 of 834 posts

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#611

Post by flarg »

swiley wrote: post #528
totaldude87 wrote: post #491

All previous major spreadsheet screwups were costing money (largest being $6 billion in losses due to a excel formula mishap) , but this one is playing with lives :( 5 – London Olympics Oversells Swimming Event by 10,000 Tickets 4- Banking powerhouse Barclay’s accidentally bought 179 more contracts than they intended in their purchase of Lehman Brothers assets in 2008. Someone hid cells containing the unwanted contra…

Is it really that hard to spend 5 minutes setting up a decent database? Use sqlite if you really want something simple. I guess you still have to understand locking (especially on distributed filesystems.) I've certainly seen people mess that up with spreadsheets.

No it isn't, but in any big org the plebs are only allowed to use Office and anything else is IT captured.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#613

Post by koliber »

totaldude87 wrote: post #491

All previous major spreadsheet screwups were costing money (largest being $6 billion in losses due to a excel formula mishap) , but this one is playing with lives :( 5 – London Olympics Oversells Swimming Event by 10,000 Tickets 4- Banking powerhouse Barclay’s accidentally bought 179 more contracts than they intended in their purchase of Lehman Brothers assets in 2008. Someone hid cells containing the unwanted contra…

They are costing money. No doubt.

At the same time, the business world runs on Excel. How much money is Excel making?

I've done my share of cursing at Excel at various jobs. At the same time, I am grateful for the quick and easy way it allows me and many others to manipulate data. It's unfair to just cite the costs of using Excel without acknowledging the benefits it brings.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#614

Post by zelphirkalt »

edanm wrote: post #589

As a data engineer, I think blaming Excel for this is the wrong framing. The real problem here is with the entire process, and specifically, the lack of testing. Excel is great for many use cases, especially if you need people to enter data somewhere. Its UI is unparalleled in terms of quickly giving something to users that they can understand, mess around with, and verify. It's a very common use case to then need to…

Excel probably excels at promoting file formats like xlsx and such. I would not be surprised, if many mistakes, including choice of xlsx when saving, came from that, instead of storing data in csv, with the most simple csv syntax. So in that regard, I think excel is sometimes just bad.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#615

Post by lbruder »

curiousllama wrote: post #510
bobbylarrybobby wrote: post #494

Earlier quoted context omitted.

The issue with is excel is that its failure mode is suboptimal. Programming languages will generally crash at runtime when an unrecoverable error occurs (index out of bounds, memory allocation failed, null pointer, etc). But excel just throws away your data?

If it fails, it’s often silent. If not silent, hidden. If not hidden, misleading, if not misleading, opaque. If not opaque, trivial. Excel is a wonderfully powerful tool that’s very bad at handling errors clearly.

It's the PHP of business software. Powers half the market, but boy do mistakes cost you.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#617

Post by WouterSpaak »

smsali97 wrote: post #616

In these cases shouldnt Microsoft Excel raise a warning that data loss may occur if youre using the legacy file format?

I believe Office will warn you when saving as .xls (or .doc instead of .docx, or .ppt instead of .pptx for that matter) that some data loss may occur. Problem is that nobody reads those warnings and just hits the OK button.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#618

Post by jl6 »

politelemon wrote: post #349
jl6 wrote: post #315

It’s fun to point and laugh, but is this really just the consequence of an unprecedented health emergency demanding a rush job? As far as bugs go, it doesn’t sound that bad. They didn’t lose data - they just processed it late? And they spotted it within days/weeks, and have a workaround/correction already? And it’s only the reporting that was wrong, not the more important part where they inform people of results? I’d…

They had a budget of GBP 36 million. It's pretty bad.

What’s your source for that budget? The error was by PHE, not Serco.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#619

Post by collyw »

koliber wrote: post #613
totaldude87 wrote: post #491

All previous major spreadsheet screwups were costing money (largest being $6 billion in losses due to a excel formula mishap) , but this one is playing with lives :( 5 – London Olympics Oversells Swimming Event by 10,000 Tickets 4- Banking powerhouse Barclay’s accidentally bought 179 more contracts than they intended in their purchase of Lehman Brothers assets in 2008. Someone hid cells containing the unwanted contra…

They are costing money. No doubt. At the same time, the business world runs on Excel. How much money is Excel making? I've done my share of cursing at Excel at various jobs. At the same time, I am grateful for the quick and easy way it allows me and many others to manipulate data. It's unfair to just cite the costs of using Excel without acknowledging the benefits it brings.

The real question should be would it be cheaper to replace it with a proper solution (probably something custom built application written by a software engineer).

Excel's ease of use is it's downfall. It is the worlds most popular database, despite not actually being a database. I have wasted countless hours dealing with Excel where something else should have been used. I built a database for a friend recently, I think 75% of the work was cleaning the existing data from the excel to get it into the database.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#620

Post by collyw »

andreareina wrote: post #569
Bukhmanizer wrote: post #546

Earlier quoted context omitted.

I think similar to “code smells” in software engineering, using excel for critical data pipeline work can be considered a “data smell”. It’s not bad in and of itself, but it’s a signal that something may have gone seriously wrong with the data engineering process.

Unfortunately excel is the lingua franca of data exchange in SME

I am sure there is a solution to this problem that could make someone a fair bit of money, but i am not quite sure what it is yet.
Post reply on HN