Live data from Hacker News

Missing Covid-19 test data was caused by the ill-thought-out use of Excel

bbc.com

741–750 of 834 posts

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#741
post #737

Earlier quoted context omitted.

As a lay-person, what would you use for non numbers tabular data?

People love to use Excel to look at (with their eyes, or a screen reader, i.e., non programmatically) all kinds of data. They use it to list exports from data stores, and then take notes on what they discovered, show and hide data using the filter option, etc. It is just far too easy to use (and misuse) that it ends up being used for any type of data that fits into a table.

I mean I thought you said that they should not use excel for these scenarios, but something else. So I thought you had an alternative software in mind?

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#742
post #619

Earlier quoted context omitted.

They are costing money. No doubt. At the same time, the business world runs on Excel. How much money is Excel making? I've done my share of cursing at Excel at various jobs. At the same time, I am grateful for the quick and easy way it allows me and many others to manipulate data. It's unfair to just cite the costs of using Excel without acknowledging the benefits it brings.

The real question should be would it be cheaper to replace it with a proper solution (probably something custom built application written by a software engineer). Excel's ease of use is it's downfall. It is the worlds most popular database, despite not actually being a database. I have wasted countless hours dealing with Excel where something else should have been used. I built a database for a friend recently, I thi…

No, because the proper solution written by a software engineer will contain more bugs.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#743
post #736

I work in healthcare and by far the biggest production issues we've ever run into are people who run Excel spreadsheets through our file processing where the leading zero's have been removed from patient identifiers because that's Excel's default behavior and you CANNOT TURN IT OFF! EDIT: I have no idea who downvoted my post because what I said is 100% true. We have to tell customers to stop opening CSVs in Excel and…

> Excel interprets a number and then formats it as a number, Unless you instruct it to interpret the field as a string. > but in healthcare, 10 digit numbers are really strings. I'm wondering, if you expect 10 digits and you get less than that, how difficult is it to add some padding zeroes?

This would assume people (many who have over a decade of experience using Excel) actually know how to do that. The largest problem with Excel is people.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#744

I work in healthcare and by far the biggest production issues we've ever run into are people who run Excel spreadsheets through our file processing where the leading zero's have been removed from patient identifiers because that's Excel's default behavior and you CANNOT TURN IT OFF! EDIT: I have no idea who downvoted my post because what I said is 100% true. We have to tell customers to stop opening CSVs in Excel and…

Couldn't you validate the data on upload or if it's such a common problem just auto-fix it by left padding w/ zeros?

Validating on upload is sensible and should be done anyway. However, left-padding is not always going to work, it depends on the data. Common problems I see are with phone numbers. UK mobiles begin 07nnn. Excel kindly drops the leading zero and then really helpfully treats the long number as being an exponential-format floating point number, which cannot be turned back to the original number. In our systems, we require country codes in front of the numbers, so UK mobiles should being 447nnn. That doesn't lose the leading zero but still gets the E-format mess when opened by Excel. Customers often miss out the 44 and use the national 07 format. If that leading zero is missing, it can be quite difficult for a system to reliable determine whether that number, beginning with a 7, is an international number for a destination in Russia, or a user error and should've been 447, from an 07 data value.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#745
post #619

Earlier quoted context omitted.

The real question should be would it be cheaper to replace it with a proper solution (probably something custom built application written by a software engineer). Excel's ease of use is it's downfall. It is the worlds most popular database, despite not actually being a database. I have wasted countless hours dealing with Excel where something else should have been used. I built a database for a friend recently, I thi…

People start using excel when the requirements are unknown, they use excel to understand the requirements while still being functional. To build a "proper solution", you frontload all the requirement discovery to build a system which only you or another developer will be able to update/change. We blame excel, but excel is really just being used for prototyping and nobody takes a decision at a certain point to move on…

I would say that people use Excel because they don't understand how to set up a relational database. It's usually non-technical users that set up Excel, then at a later stage I get asked to create a proper database from it.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#746
post #609

Earlier quoted context omitted.

Perhaps by "correct way" you meant "dodgy hack to make Excel happy and risk breaking more sensible implementations"? Excel may predate the RFC but AFAIK MS didn't invent or coin the term CSV, so you can't just say whatever Excel does is correct. The RFC is loose because of nonsense like this, it doesn't mean it was ever a good idea.

Do you want to be a useless sophist or develop software?

The reason you use CSV is that you want to use it with software that ISN'T Excel. Otherwise you'd just use .xlsx. No other software uses this convention, and it's not correct CSV.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#747

I work in healthcare and by far the biggest production issues we've ever run into are people who run Excel spreadsheets through our file processing where the leading zero's have been removed from patient identifiers because that's Excel's default behavior and you CANNOT TURN IT OFF! EDIT: I have no idea who downvoted my post because what I said is 100% true. We have to tell customers to stop opening CSVs in Excel and…

Down-voted as it is factually incorrect. See comment by reportgunner below.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#748
post #592

Earlier quoted context omitted.

> I have no idea who downvoted my post That would be the Excel devs working at Microsoft. They read HN. I can feel it.

Excel devs, do you hear me? Repent for your sins! Go back to Microsoft and fix those problems. The cost of Excel mishandling numbers and dates must be many orders of magnitude higher than your lifetime salary. Thank you.

They can't. All those spreadsheets that depend on that behavior are out there. It makes spell create with an e look easy.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#749
post #694
post #635

Earlier quoted context omitted.

The problem with a database is that business users struggle to build them, they take longer, and sometimes can’t hold logic in the same way. Plus in a database you can’t do the same sort of real-time analysis and also pass the document around for other non-technical folk to add to and modify. In the real world in big companies, people often don’t want to talk to IT because they over-spec and quote what are perceived…

To add to this point, the latest article says that they started building a replacement to the spreadsheet two months ago, so I think that reinforces the speed of a spreadsheet implementation vs anything else in the public sector!

You might not have realized this but they should have started work in January.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#750
post #737

Earlier quoted context omitted.

People love to use Excel to look at (with their eyes, or a screen reader, i.e., non programmatically) all kinds of data. They use it to list exports from data stores, and then take notes on what they discovered, show and hide data using the filter option, etc. It is just far too easy to use (and misuse) that it ends up being used for any type of data that fits into a table.

I mean I thought you said that they should not use excel for these scenarios, but something else. So I thought you had an alternative software in mind?

I use Splunk (our organization pays lots of money for it, and yet it just sits there because people would rather use Excel). There is also Tableau and other products like it. Most of these tools have a free version, as well. They take some learning, but it is well worth it. Of course, taking that same amount of time to actually learn how Excel works is worth the time, too. I guess the issue is that Excel is just too "easy" to use, and hence people using it for years and years and not actually knowing how to use it.

I am not trying to be an elitist about this. It is just that the misuse of Excel (because people do not know how to use it) causes massive issues on a daily basis.

Post reply on HN