Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

One in five genetics papers contains errors thanks to Excel (2016)

science.org

Post reply on HN
Print view View original

241–250 of 267 posts

Re: One in five genetics papers contains errors thanks to Excel (2016)

#241

Post by fshr »

tssva wrote: post #234
mark-r wrote: post #228

Earlier quoted context omitted.

I would wager more people open a CSV by double-clicking on it rather than using the import data wizard. And even if you use the wizard it takes extra work to specify the type of each column, which most people won't bother with. Writing some VBA is a simple process if you're a programmer. I wonder how many genetic researchers fit that description? P.S. when I said "too late to fix it", I meant by some process within E…

Expecting you to learn the basics about the tools you're using is not expecting too much. And if you are too lazy to spend a few seconds specifying data types then you get what you deserve.

> Expecting you to learn the basics about the tools you're using is not expecting too much.

Do you/have you worked in a corporate environment? You seem to have an idealistic view about how end users are expected to use Excel.

Re: One in five genetics papers contains errors thanks to Excel (2016)

#242

Post by anthk »

fifilura wrote: post #236

I always felt that there is a hierarchy for working with numbers that goes something like this Ease of use: 1. Excel 2. SQL 3. Functional programming (e.g. Scala, Python to some measure e.g. Pandas) 4. Imperative programming (C/C++/Java) But then there another hierarchy that (roughly) goes in the other direction, which is about quality, repeatability, tooling. If you are at 1 or 2, you responsibility will not be abou…

2 s,SQL,j

https://www.jsoftware.com/indexno.html

Also:

https://www.jsoftware.com/help/dictionary/intro.htm

EDIT: The help section has 6 books. If you want, you can do self-teach yourself advanced math stuff with very few lines. I suggest to install Gnuplot as a dependency, for plots.

Re: One in five genetics papers contains errors thanks to Excel (2016)

#243

Post by jimnotgym »

remus wrote: post #126
jimnotgym wrote: post #124

Fixing the headline... One in five genetics papers have errors caused by mistakes in the use of Excel As if longhand calculations never have errors?

The difference here is that the excel errors are systematic so should be more easily avoidable, whereas long hand calcs are more varied and harder to correct.

I use Excel a lot. These things don't trip me up in my field. Could it be that this field is a rather niche use case, and therefore the users should learn to watch out for them?

Re: One in five genetics papers contains errors thanks to Excel (2016)

#244

Post by fifilura »

anthk wrote: post #242
fifilura wrote: post #236

I always felt that there is a hierarchy for working with numbers that goes something like this Ease of use: 1. Excel 2. SQL 3. Functional programming (e.g. Scala, Python to some measure e.g. Pandas) 4. Imperative programming (C/C++/Java) But then there another hierarchy that (roughly) goes in the other direction, which is about quality, repeatability, tooling. If you are at 1 or 2, you responsibility will not be abou…

2 s,SQL,j https://www.jsoftware.com/indexno.html Also: https://www.jsoftware.com/help/dictionary/intro.htm EDIT: The help section has 6 books. If you want, you can do self-teach yourself advanced math stuff with very few lines. I suggest to install Gnuplot as a dependency, for plots.

Very good! I believe you.

Unfortunately I feel like it would be irresponsible to transition our stack to working with J because of available competence and relearning.

I can see it being used in research though.

Re: One in five genetics papers contains errors thanks to Excel (2016)

#245

Post by kevin_thibedeau »

kstrauser wrote: post #217
kqr wrote: post #167

Earlier quoted context omitted.

Except 3 - 2 ^ makes it hard to express 3 2 - ^ without accidentally subtracting, so in this case a unary negation sign needs to be a different symbol!

I dug out my HP 50g over the weekend to play with, and conveniently… Sigh. I understand why we commonly enter math on basically a teletype-with-ASCII, and I don’t have an urge to go all APL, but for a while we were so close to a future where we could’ve had separate negation or multiplication or exponentiation symbols that might’ve removed so much room for error. I mean, that little calculator and its predecessors we…

It would be nice if OSs defaulted numeric keypad / * and - to Unicode ÷, ×, −. I never use them even when I do use the digits. That would solve the more glaring typewriter legacies. Then you'd just have the apostrophe/single-quote as the last remaining unification.

Re: One in five genetics papers contains errors thanks to Excel (2016)

#246

Post by paledot »

XlA5vEKsMISoIln wrote: post #235
tssva wrote: post #224

Earlier quoted context omitted.

The specific import process isn't some sort of esoteric process. It is the data import wizard. Also as I said if you are regularly importing data from a file with the same format writing some VBA to do so is pretty quick and simple task. Also your data isn't gone. It is still in the CSV file you imported it from. Re-import it.

Excel chews up CSVs that it opens. I know this because an accountant checked each file our code produced using Excel before trying to import it into another program. We proofread our code before we realizing the problem was somewhere else. Shoulder-surfed the process, found the giant bug with a green X on it.

It's no better at exporting to CSV. I wrote a CSV parser a few years ago that had one set of logic for Excel CSVs and a completely different set for everything else.

Re: One in five genetics papers contains errors thanks to Excel (2016)

#247

Post by lotsoweiners »

cynicalsecurity wrote: post #212
lbriner wrote: post #161

Don't get me started on the automatic conversion of strings that it thinks are in scientific notation into numbers - which you can't switch off! We have large data exports from systems that include things like unique location code. You accidentally happen to notice that a block of these look weird and it isn't just the display of them that has changed, the contents of the cell were changed by Excel automatically, wit…

Why still use MS Office when LibreOffice is freely available?

Because that is what your work gives you and what all of your coworkers use.

Re: One in five genetics papers contains errors thanks to Excel (2016)

#248

Post by setopt »

zmgsabst wrote: post #138
setopt wrote: post #114

Earlier quoted context omitted.

This is a good way to think of it. It would be very confusing if “0 - 3^2” and “0 + (-3^2)” should mean different things.

But what about (0-3)^2? You said that “-3” = “0-3”. So we have “-3^2” is “(0-3)^2” is 9. Agreeing with -3^2 = 9. You’re performing a sleight of hand when you define “-3” to be “0-3”, but move the parenthesis to get your second equation. You have to insert your definition as a single term inside parenthesis — you can’t simply remove them to change association (as you have done). That’s against the rules. So if you thi…

> You said that “-3” = “0-3”.

No, I didn’t.

Re: One in five genetics papers contains errors thanks to Excel (2016)

#249

Post by tssva »

fshr wrote: post #241
tssva wrote: post #234

Earlier quoted context omitted.

Expecting you to learn the basics about the tools you're using is not expecting too much. And if you are too lazy to spend a few seconds specifying data types then you get what you deserve.

> Expecting you to learn the basics about the tools you're using is not expecting too much. Do you/have you worked in a corporate environment? You seem to have an idealistic view about how end users are expected to use Excel.

I worked in corporate environments for decades.

The original comment I responded to said they regularly imported large data sets and the in the case of the genetists they also are regularly importing data into Excel. In other words Excel is a regularly used and fundamental tool to their work. In this case I would expect someone to learn the basics of using it. Just as I would expect a developer to learn their editor, build system, version control system, etc.

Re: One in five genetics papers contains errors thanks to Excel (2016)

#250

Post by tssva »

XlA5vEKsMISoIln wrote: post #235
tssva wrote: post #224

Earlier quoted context omitted.

The specific import process isn't some sort of esoteric process. It is the data import wizard. Also as I said if you are regularly importing data from a file with the same format writing some VBA to do so is pretty quick and simple task. Also your data isn't gone. It is still in the CSV file you imported it from. Re-import it.

Excel chews up CSVs that it opens. I know this because an accountant checked each file our code produced using Excel before trying to import it into another program. We proofread our code before we realizing the problem was somewhere else. Shoulder-surfed the process, found the giant bug with a green X on it.

Excel doesn't change CSV files when it imports them. If the imported file was being changed then the user was saving back to the same file they imported from.
Post reply on HN