In Norway there was a $14 billion calculation error based on Excel, that resulted in opening of new oil fields in the Barents Sea:
https://translate.googleusercontent.com/translate_c?depth=1&...
Missing Covid-19 test data was caused by the ill-thought-out use of Excel
381–390 of 834 posts
Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel
#382Earlier quoted context omitted.
Have you every worked in a large enterprise? I have personally tested an Excel based credit rating tool, to be rolled-out globally by a major financial institution. The fact that one should not do this, is by no means a reason not to do it.
Fun fact: Toshiba has software that generates entire CRUD app skeletons driven 100% by excel spreadsheets. You fill out object definitions in excel spreadsheets with property names and data types etc. It's like an extremely uncool long lost cousin of rails -g Source: Got one of their engineers to show me after I heard about it and had to know if it was real. Anyone who's ever worked with or in Japan knows just how fa…
to play devil's advocate, i had a desktop support job in college and worked with an executive assistance. She was _very_ attractive and so i was at her desk "working on the printer" a lot, she knew Excel better than anyone i've ever seen (before or since). It was like watching a magician.
In the right hands Excel is pretty amazing.
Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel
#383Earlier quoted context omitted.
It's not a user issue, it's a government incompetence issue. Who was the technical genius who thought 'right, we need to store thousands of lines of critical and important data, I know! Excel!'
Alternatively... "We need to store some data for a virus" "Where does the data come from?" "Every hospital sends us an excel sheet" "Well, let's merge all of it into a bigger sheet and display it or export it"
>"We need to store some data for a virus" "Where does the data come from?" "Every hospital sends us an excel sheet" "Well, let's merge all of it into a bigger sheet and display it or export it"
you have no idea how right you are :)
Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel
#384Earlier quoted context omitted.
I think you'd be shocked by how many of our financial and social institutions rely on emailing Excel spreadsheets to one another.
Don't forget healthcare! Excel and CSV for tons of data exchange.
I think the healthcare industry is probably the only thing keeping fax machines alive. Interestingly, fax machine are tagged as HIPPA compliant. Lots of spreadsheets get printed and then fed to the fax machine for transmission instead of email/whatever. On the other side, and i'm being serious, someone gets the fax and creates a new spreadsheet and enters all the rows manually. fun fun.
Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel
#385Earlier quoted context omitted.
Excel never works for math, at any scale. It's a constant trashfire of "looks fine" until it shows a result someone doesn't want, and then an error is found that changes results and the cycle repeats. Excel is nice for visualizing and browsing data you already have, and informally searching and sorting for hypothesis generation.
My 4th order Runge-Kutta ODE solver I did in an afternoon in Excel the other day, because I didn't want to pirate Matlab to do a homework assignment, begs to differ :). Not to mention, the entire financial industry and many (most?) small businesses. Lots of people use Excel for various types of math, and it works fine, if you know what you're doing . Excel not being idiot-proof doesn't mean it's not usable. And that'…
> Not to mention, the entire financial industry
That IS the dumpster fire. Trust me, the reason I know is that I used to be that guy who thought Excel was a great tool.
I built derivatives spreadsheets, backoffice spreadsheets, trading systems with realtime data, all sorts of crap in Excel.
Really, it's Stockholm Syndrome. People who previously had zero computing power at their disposal think they've found the hammer that solves all problems when they're introduced to Excel, because now they can calculate lots of numbers.
They just get blinded by the revelation that they can now calculate "anything" and are happy to pay whatever the cost is in terms of future maintenance, ease of understanding, etc.
> Excel not being idiot-proof doesn't mean it's not usable.
This is absolutely true, I'd be able to do much better with Excel now than earlier. The problem is Dunning Krueger. There are too many people who think they can code once they're able to get a bit of Excel going, and they don't know that they can't. Not trying to be condescending, I've been there myself. It's just that you get a lot of "coding is a thing I have to do in order to get to some target", and so people think that once they've finally bashed out their spreadsheet, they've figured it all out.
Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel
#386Oh, Excel, how I hate to love you...
As is the case for many, Excel is one of my main tools. From financials to data gathering to electrical, mechanical or software engineering, Excel has always been there. It is fair to say I have made lots of money thanks to this tool.
And yet, every so often...
Many years ago a rounding error in a complex Excel tool we wrote to calculate coefficients for an FPGA-based polyphase FIR filter cost us a little over six months of debug time. I still remember the "eureka!" moment at two in the morning --while looking at the same data for the hundredth time in complete frustration-- when I realized we should have used "ROUNDUP()" rather than "ROUND()".
Most recently, I was working with a client who chose to build a massive Excel sheet to gather a bunch of relevant data. The person doing the work seems to think they know what they are doing (conditional formatting and filtering don't make you an expert). This poor spreadsheet has every color in the rainbow and a mess of formulas. It's impossible for anyone but the guy who created it to touch it.
Here's a hint:
Do not mix data with presentation. Where have we heard that before?
This is one of my pet peeves with Excel. If you need to gather a bunch of data, do it. Treat Excel like a database (apply normalization if you can!) and keep it with as little formatting as you possibly can. Then do all the formatting and calculations on a "Presentation" sheet or sheets. Just don't pollute your database with formatting.
EDIT: Thinking about the UK problem, if they were working with a ".xlsx" file and accidentally saved it in ".xls" form, well, as they say, "There's a warning dialog for that".
What surprises me the most about these kinds of incidents is that people keep working on the same single document. In other words, no semblance at all of what the software business knows as version control.
Decades ago I adopted the idea that storage is cheap and always getting cheaper. If I am working on something critical, I never work more than one day without a backup. I make a copy and continue editing. In most cases I make a new copy every single day.
Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel
#387It’s fun to point and laugh, but is this really just the consequence of an unprecedented health emergency demanding a rush job? As far as bugs go, it doesn’t sound that bad. They didn’t lose data - they just processed it late? And they spotted it within days/weeks, and have a workaround/correction already? And it’s only the reporting that was wrong, not the more important part where they inform people of results? I’d…
There's a test and trace system that is supposed to phone everyone with a positive test result and get details of all their close contacts (people they sat in an office with, restaurants they visited, etc), who are in turn contacted.
A delay in the test results means the contact tracing could not happen, and people will have been going around spreading the virus they caught off the original, known cases.
Also, the whole test and trace system has been in the news a lot recently here for various failings, and things like this will just further knock people's confidence in it.
Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel
#388The actual title of the article is awfully clickbaity and misleading. As i understand the issue is not at all Microsofts fault, but NHE using an inadequate way of storing the data
Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel
#389It’s fun to point and laugh, but is this really just the consequence of an unprecedented health emergency demanding a rush job? As far as bugs go, it doesn’t sound that bad. They didn’t lose data - they just processed it late? And they spotted it within days/weeks, and have a workaround/correction already? And it’s only the reporting that was wrong, not the more important part where they inform people of results? I’d…
They had a budget of GBP 36 million. It's pretty bad.
Serco received 45.8 Million in March (not disclosed until April), and I'm pretty sure they've had more since.
Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel
#390Earlier quoted context omitted.
CSV != Excel. I work CSVs regularly and can't recall the last time I opened up Excel (intentionally)
Doesn't Excel by default capture the .csv extension so that it gets called automatically when you try to open the file? Since Excel is one of the few standard pieces of software that knows how to open CSV, it gets used a lot of times when it shouldn't. There's another post I made comparing Excel to a swiss army knife, and there's a reason for that.
I don't know many engineers - even in the data team - with an Office license.