Live data from Hacker News

Missing Covid-19 test data was caused by the ill-thought-out use of Excel

bbc.com

831–834 of 834 posts

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#831

Earlier quoted context omitted.

Every single problem I've ever had with excel stems from excel trying to be too clever. Nearly all my problems with importing data into it could be solved if excel left everything as is until I asked it to format values a certain way.

And the people who make excel will just tell you "excel is not a database" lol

Which sadly, is what I want to tell them too!

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#832

Earlier quoted context omitted.

Couldn't you validate the data on upload or if it's such a common problem just auto-fix it by left padding w/ zeros?

These aren't fixed-width fields with a known length. The length of the field is unique to the person sending the file and they don't tell us how long it's supposed to be.

The parent post says: "Excel interprets a number and then formats it as a number, but in healthcare, 10 digit numbers are really strings." - so I was under impression that the length for that particular column is fixed - probably just misunderstood the sentence.

I wonder how human operators figure out if the value is correct, or the Excel messed it up, or the input was invalid in the first place? If it's even possible to do it reliably then probably there is some set of patterns and methods that possibly could be turned into an algorithm... just thinking out loud here, but seems as an interesting problem to tackle...

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#833

I work in healthcare and by far the biggest production issues we've ever run into are people who run Excel spreadsheets through our file processing where the leading zero's have been removed from patient identifiers because that's Excel's default behavior and you CANNOT TURN IT OFF! EDIT: I have no idea who downvoted my post because what I said is 100% true. We have to tell customers to stop opening CSVs in Excel and…

Interesting to see just how many responses to this real-world problem are suggesting that people rewrite CSV files to work with Excel. They clearly do not understand system integration and the use of CSV text files for data interchange between multiple systems and application. Hey, JSON and Javascript libraries are the answer to that, eh There are already enough potential issues with CSV interpretation on wrapping st…

[deleted]

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#834

I work in healthcare and by far the biggest production issues we've ever run into are people who run Excel spreadsheets through our file processing where the leading zero's have been removed from patient identifiers because that's Excel's default behavior and you CANNOT TURN IT OFF! EDIT: I have no idea who downvoted my post because what I said is 100% true. We have to tell customers to stop opening CSVs in Excel and…

> there isn't a way to get them back if you save/overwrite. Of course there is, in your case. If patient identifiers have a fixed, known length, then you can pad with leading zeros to recover them. You only have a problem if 012345 and 12345 are distinct patient identifiers. It is bone-headed in the first place to use numeric-looking identifiers (such as containing digits only) which are really strings, and then allo…

The lengths of the identifiers are not known, man. We are dealing with like 200 hospitals. They all have various kinds of Electronic Medical Record system like Epic or Cerner, or one of the 15 other third party vendors that all have custom implementations. Healthcare is insane in our country.
Post reply on HN