Live data from Hacker News

Missing Covid-19 test data was caused by the ill-thought-out use of Excel

bbc.com

251–260 of 834 posts

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#252
post #242

I hate solving problems and not knowing how I did it, so this particular experience has stuck in my craw. Trying to develop a budget to pay off debts, my partner made this elaborate Excel spreadsheet and the output was that basically she had no spending money and I had very little, until one or both of us got a raise. It was far more austere than either of us were willing to go. So I started over using a different eq…

Your post is so stupid that someone could make a copy-pasta of it, e.g. "I tried to write a program in C++ but it didn't want to compile, so I spent 90 minutes poking at the code, looking for the error and never did find it. I don't know what the right solution is to this sort of problem, at least for developers. ------------------- "I don't know how to use Excel (most popular spreadsheet program used for budgeting),…

This reminds me of developers talking about the git CLI. At some point you have to stop blaming the users.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#253
post #242

Earlier quoted context omitted.

Your post is so stupid that someone could make a copy-pasta of it, e.g. "I tried to write a program in C++ but it didn't want to compile, so I spent 90 minutes poking at the code, looking for the error and never did find it. I don't know what the right solution is to this sort of problem, at least for developers. ------------------- "I don't know how to use Excel (most popular spreadsheet program used for budgeting),…

Umm, who is qualified to use Excel if both scientists and economists frequently have errors in their published works? https://www.economist.com/graphic-detail/2016/09/07/excel-er...

Umm, who is qualified to write programs in any programming language, if both programmers and developers frequently have errors in their code?

https://cve.mitre.org/cve/

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#254
post #16

We did a bit of work for laboratories this year and csv is not an uncommon exchange format between labs. In general almost all exchange formats are text based, with labs saying they will upgrade to „modern xml formats“ at some point in the future. So seen in this context a csv or an excel file doesn‘t really surprise me and should probably also be seen in this context.

XML would fix the max rows issue, but open you up to OOM issues instead!

Sorry, but if you run into OOM issues by parsing an XML file, you're using the wrong API.

The DOM for a large XML document will of course take tons of space in memory. The key to parsing XML files quickly and with low memory consumption is to only keep in memory what's necessary, by streaming over the elements.

https://en.wikipedia.org/wiki/Simple_API_for_XML

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#255
post #250

Earlier quoted context omitted.

You made two different spreadsheets, they came up with different results, and you're blaming Microsoft for the error? Really? Excel, whether you like it or loathe it, is in such wide use around the world that $300 math errors would have been noticed a very long time ago. I could believe that there are still many lurking bugs with obscure corner cases, nasty floating point rounding minutiae and so on, but I would bet…

I'm not so sure he claimed that Excel can't add and subtract numbers. I think the argument is that even two people can make such a mess of a spreadsheet that nobody can really untangle it later.

That's about the size of it.

Tack on that developers have other, better options, but if you're not a developer I don't have a good solution for you, and that I don't like that state of affairs.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#257

I hate solving problems and not knowing how I did it, so this particular experience has stuck in my craw. Trying to develop a budget to pay off debts, my partner made this elaborate Excel spreadsheet and the output was that basically she had no spending money and I had very little, until one or both of us got a raise. It was far more austere than either of us were willing to go. So I started over using a different eq…

You made two different spreadsheets, they came up with different results, and you're blaming Microsoft for the error? Really? Excel, whether you like it or loathe it, is in such wide use around the world that $300 math errors would have been noticed a very long time ago. I could believe that there are still many lurking bugs with obscure corner cases, nasty floating point rounding minutiae and so on, but I would bet…

But they aren't noticed. The classic blunder is dropping the last item off of a summation, and nobody notices until they add another item and things look janky.

Those happen all the time, we just only hear about it when the consequences are outsize.

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#258
post #242

Earlier quoted context omitted.

Your post is so stupid that someone could make a copy-pasta of it, e.g. "I tried to write a program in C++ but it didn't want to compile, so I spent 90 minutes poking at the code, looking for the error and never did find it. I don't know what the right solution is to this sort of problem, at least for developers. ------------------- "I don't know how to use Excel (most popular spreadsheet program used for budgeting),…

This reminds me of developers talking about the git CLI. At some point you have to stop blaming the users.

I dont know what is your point? Defending guy who wrote an useless story, how he used Excel once, didn't know what he is doing so something went wrong and then his conclusion is that Excel sucks?

Millions of people use the program every day, it has its own fair share of problems (like everything), but if random Karens in any corporation can learn how to use it, then you should expect the same from a random programmer on hacker news.

Also, the irrelevancy of the story is comical ("I dont like it, so it must be bad!")

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#259
post #16

We did a bit of work for laboratories this year and csv is not an uncommon exchange format between labs. In general almost all exchange formats are text based, with labs saying they will upgrade to „modern xml formats“ at some point in the future. So seen in this context a csv or an excel file doesn‘t really surprise me and should probably also be seen in this context.

> exchange formats are text based, with labs saying they will upgrade to „modern xml formats“ Worth noting that XML is also a text format. SGML even can treat CSVs as markup. There's nothing wrong with CSVs/TSVs anyway - it's a concise tabular format using only minimal special coding for a record and a field separator, as envisioned by ASCII and EDIFACT. The problem seems more like that there was no error checking in…

I feel like text-based is colloquial shorthand for "schemaless" or "dynamically typed".

Re: Missing Covid-19 test data was caused by the ill-thought-out use of Excel

#260

People keep putting the blame on Excel but it supports more columns than most implementations of SQL databases. MySQL for example supports only 4096 columns. SQLite defaults to 2000. That's way less than Excel's 16384.

True, but they don't need to relate 16384 types of data. It sounds like they were using one column per record, rather than one row per record. So if they had a sensible data model 2000 columns should be more than sufficient.
Post reply on HN