Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Missing Covid-19 test data was caused by the ill-thought-out use of Excel

bbc.com

Post reply on HN
Print view View original

491–500 of 834 posts

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#491

Post by totaldude87 »

All previous major spreadsheet screwups were costing money (largest being $6 billion in losses due to a excel formula mishap) , but this one is playing with lives :(

5 – London Olympics Oversells Swimming Event by 10,000 Tickets

4- Banking powerhouse Barclay’s accidentally bought 179 more contracts than they intended in their purchase of Lehman Brothers assets in 2008. Someone hid cells containing the unwanted contract instead of deleting them.

3-utsourcing specialists Mouchel had to endure a £4.3 million profits write down due to a spreadsheet error in a pension fund deficit caused by an outside firm of actuaries

2- Canadian power generator TransAlta suffered losses of $24 million as the result of a simple clerical error which meant they bought US contracts at higher prices than they should hav

and the Biggest one is. -

Basic Excel flaws and incorrect testing led to JP Morgan Chase losing more than $6 billion in their London Whale disaster.

https://floatapp.com/us/blog/5-greatest-spreadsheet-errors-o...

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#492

Post by kelnos »

rossdavidh wrote: post #427

While I am not in the habit of defending Microsoft, I think it should be pointed out that it wasn't using "Excel" that was the biggest problem, but using "13-years obsolete format for Excel". Not that there was any reason to be using Excel for this, but it's a bit unfair to blame Microsoft for something they never claimed XLS could do, and provided a much better capacity in XLSX 13 years ago. Again, it's hard to cry…

Sure, and I think at least the headline ("ill-thought-out use of Excel") avoids blaming MS or Excel itself.

Ultimately tools are built for particular things, and if you choose to use a tool for something it's not built for, and it breaks catastrophically, that's on you.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#493

Post by gxs »

konjin wrote: post #422
gxs wrote: post #334

Earlier quoted context omitted.

Not to mention the "system" solution will have to go through its own debugging and will undoubtedly have bugs in it for some time after initial release. On top of that, there is STILL room for human error as a user is still the one actually inputting data and designing the experiment/data flow. I'm going to get mauled on this forum given the audience, but parent comment reeks of the technical elitism on this forum an…

The same way that engineers saying you shouldn't use dung to build houses are guilty of architectural elitism. Excel is the wrong tool because it mixes presentation and data, to the point that geneticists had to change the names of genes so their spreadsheets would stop turning them silently into dates and then destructively changing the original data[0]. >Excel works great - I've personally built some very complicat…

And I've cleaned up after many, many shitty engineers, one example being engineers who want to use a sledge hammer to drive a finishing nail and again, that's just one example.

And if you want to get personal to that level they all the same know it all personality who think they write MUCH better code than they actually do.

Pretty sure they cost companies even more money.

Luckily for me, I have the practical sense to know when to use what tool and so far it's worked out just fine. I wouldn’t use excel where not appropriate, so kindly don’t make assumptions.

But more power to you, keep shitting on tools that are more sophisticated than you probably think.

People who half know excel are the worse, they are the ones who know it just well enough to think it doesn't work at all.

Custom solutions are great - but aren't always the answer and with proper process Excel will work just fine in many use cases that you might not expect.

I could blab on about this forever, because, you know, there is a lot of gray area or whatever.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#494

Post by bobbylarrybobby »

rossdavidh wrote: post #427

While I am not in the habit of defending Microsoft, I think it should be pointed out that it wasn't using "Excel" that was the biggest problem, but using "13-years obsolete format for Excel". Not that there was any reason to be using Excel for this, but it's a bit unfair to blame Microsoft for something they never claimed XLS could do, and provided a much better capacity in XLSX 13 years ago. Again, it's hard to cry…

The issue with is excel is that its failure mode is suboptimal. Programming languages will generally crash at runtime when an unrecoverable error occurs (index out of bounds, memory allocation failed, null pointer, etc). But excel just throws away your data?

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#495

Post by gxs »

ccktlmazeltov wrote: post #430
gxs wrote: post #334

Earlier quoted context omitted.

Not to mention the "system" solution will have to go through its own debugging and will undoubtedly have bugs in it for some time after initial release. On top of that, there is STILL room for human error as a user is still the one actually inputting data and designing the experiment/data flow. I'm going to get mauled on this forum given the audience, but parent comment reeks of the technical elitism on this forum an…

I'm sorry but you seem to not understand what data integrity is if you think Excel is a perfectly acceptable solution here.

It's all about the use case, man.

Within its constraints excel is great. Depending on the nature of your data and exactly what you are trying to do it can accomplish quite a bit.

The line between when to use excel and when to build custom stuff is not as clear as it seems, is all I'm trying to say.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#496

Post by gxs »

syshum wrote: post #442
konjin wrote: post #422

Earlier quoted context omitted.

The same way that engineers saying you shouldn't use dung to build houses are guilty of architectural elitism. Excel is the wrong tool because it mixes presentation and data, to the point that geneticists had to change the names of genes so their spreadsheets would stop turning them silently into dates and then destructively changing the original data[0]. >Excel works great - I've personally built some very complicat…

>>>I've personally cleaned up after people like you, Yep... The critical question that always gets blank stares is "How is this data and calculations validated" There is never an answer to that question from the ExcelMaster, it is always a variation of "it looks correct to me" Wonderful....

Except you can do it if you know what you're doing.

No one is saying to manage millions of records etc.

But you can get more mileage out of excel than people think - and people who half understand it are the worse because they know enough to know that it doesn't work.

Custom solutions are great - but aren't always the answer and with proper process Excel will work just fine in many use cases.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#497

Post by bobbylarrybobby »

codeulike wrote: post #335
jl6 wrote: post #315

It’s fun to point and laugh, but is this really just the consequence of an unprecedented health emergency demanding a rush job? As far as bugs go, it doesn’t sound that bad. They didn’t lose data - they just processed it late? And they spotted it within days/weeks, and have a workaround/correction already? And it’s only the reporting that was wrong, not the more important part where they inform people of results? I’d…

Good points Partly I think the Excel bit is news because Excel (and mistakes made with Excel) is easily relatable to people. But bugs always come up and most other bugs are just as stupid. If it had been an off-by-one loop error in some C code somewhere it would be just as dumb but you'd get none of the facepalm memes all over Twitter.

Off by one errors are all alike; every excel error is an error in its own way.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#498

Post by WoodenChair »

shadowgovt wrote: post #162
TwoBit wrote: post #157

Earlier quoted context omitted.

And what do they use instead of Excel? LibreOffice? Some expensive custom database software?

Yep, that's the problem. There's a gap in the space between the generality of Excel and the performance of a SQL DB with no UI that nobody wants to fill. Microsoft Access sits there, but nobody uses it for some reason.

That's what Filemaker is supposed to be.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#499

Post by theptip »

segfaultbuserr wrote: post #436
theptip wrote: post #238

Earlier quoted context omitted.

> There's nothing wrong with CSVs/TSVs I can see the theoretical purity of this statement, but based on my experience working with CSV files generated by actual non-technical users I have to disagree here. There are a number of footguns here that are really subtle and the average non-technical user has no hope of spotting them. Problems that I've seen in the wild, off the top of my head: * Windows vs. Linux line term…

Almost none [0] of the problem, such as line terminators, encodings, or escape characters, is an inherent flaw in the idea of using a text-based file to represent data. They can be fixed by just a little bit of standardization. But unfortunately, I don't think it's possible, we're stuck at here. [0] Except type conversion, which is a real problem.

Agreed; I think the crux of the problem is that CSV is not a format that supports metadata. And even if you somehow created such a convention/standard, it's impossible to upgrade all the existing users with 10+ year-old Excel installs; that ship has already sailed.
Post reply on HN