Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An alarming number of scientific papers contain Excel errors

washingtonpost.com

Post reply on HN
Print view View original

21–30 of 74 posts

Re: An alarming number of scientific papers contain Excel errors

#21

Post by cm2187 »

7952 wrote: post #19
ramblenode wrote: post #11

Earlier quoted context omitted.

I would counter with "use the right tool for the job." Excel is great at churning out fast and dirty estimates for low impact work. The problem is when it's used for large, complex, or important problems because these are just not Excel's domain--something obvious when looking at the kinds of features and bug fixes MS has prioritized over the years.

It is damning that you cannot use a basic computer tool for anything that is large, complex, or important. Isn't that exactly what computers are for? Call me idealistic but I want a spreadhseet that can handle milions of rows. Maybe add a "strict" mode to keep people out of trouble.

You could have an option to disable any formatting in a tab, and then Excel should be able to handle a billion rows in that tab. But you can't have all the flexibility of Excel on a massive scale. Even a laptop today is insanely fast and has an insane amount of memory.

Re: An alarming number of scientific papers contain Excel errors

#22

Post by Rapzid »

ramblenode wrote: post #11
krapht wrote: post #5

Earlier quoted context omitted.

I'll just quote Soustrup here: "There are only two kinds of languages: the ones people complain about and the ones nobody uses." In particular you see the spreadsheet GUI as a downfall, but I think it is a great enabler that has allowed millions of non-programmers to create useful programs.

I would counter with "use the right tool for the job." Excel is great at churning out fast and dirty estimates for low impact work. The problem is when it's used for large, complex, or important problems because these are just not Excel's domain--something obvious when looking at the kinds of features and bug fixes MS has prioritized over the years.

Excel is used HEAVILY at all levels of business for large, complex, and important things. I'm curious what this right tool is that they should be using instead.

Re: An alarming number of scientific papers contain Excel errors

#23

Post by madaxe_again »

ramblenode wrote: post #3

MS Excel is absolutely unfit for most scientific and engineering problems. The spreadsheet GUI, lack of good version tracking/history, and eagerness to coerce data types and "correct" values makes it easy to introduce errors that will go unrecognized and propagated through calculations. Unfortunately this story just keeps repeating itself. But all of this is just a secondary concern to Excel's real trouble: it's hist…

At university, we were forbidden from using Excel, and only FORTRAN 77 was allowed. At the time, this seemed strange and anachronistic, but when you consider that it had been in use for 25 years at the time, and was a known quantity that didn't have strange and unknown bugs, it actually seems sensible in retrospect.

I do however recall being mightily annoyed that I was having to calculate ephemera in "arcane bullshit", so I can understand why many shortcut to excel.

Experience makes fools of our past selves.

Re: An alarming number of scientific papers contain Excel errors

#24

Post by cm2187 »

dagaci wrote: post #12

First I thought this story was about math and numerical errors. But it's actually about auto- formatting and auto-correction. "Excel automatically converting gene names to things like calendar dates or random numbers" In this case, I think what is needed is some kind of rudimentary knowledge of data-types. Or perhaps more simply a scientific template which is actually plain text by default. But how are people not not…

The worst with these formatting is that Excel will behave differently in different regions. Another headache in large international corporations/collaborations.

Re: An alarming number of scientific papers contain Excel errors

#25

Post by keithpeter »

https://help.libreoffice.org/Calc/Deactivating_Automatic_Cha...

Type apostrophe at beginning of the gene name ('MARCH1) or format the column for gene names as text (click column letter, then Format | Cell and select text)

If people want to use a spreadsheet application for this kind of data collection (and that is a big if I think) then they perhaps need to have some agreed lab protocols for setting up and checking the spreadsheets. This is a known issue in financial circles...

http://www.eusprig.org/basic-research.htm

Re: An alarming number of scientific papers contain Excel errors

#26

Post by pbhjpbhj »

Noseshine wrote: post #17
Steeeve wrote: post #9

Is everybody a washington post subscriber? Or have I missed the route around the paywall somehow?

I read all my WP articles via right-click "Open link in incognito window". (OT now:) If anyone thinks that's cheating them out of money - the choices are not "read for free" and "read for the appropriate price", the choices are "read for free" or "don't read". The reason I read is to procrastinate, so the value I get out of it is actually negative (same with HN...). Even their occasionally excellent articles (like th…

>The reason I read is to procrastinate, so the value I get out of it is actually negative (same with HN...). //

Entertainment has value doesn't it? It's not a simple financial value like the simplistic use of opportunity cost as being the price you could bill those hours at, but it's a useful and functional part of being human.

I don't have a problem with you reading something people broadcast to the public internet though.

Re: An alarming number of scientific papers contain Excel errors

#27

Post by pbhjpbhj »

hirenj wrote: post #4

It's funny to consider that these errors slipped past the peer review stage. It really highlights the major issue with reviewing source code published as part of an analysis. If there aren't enough resources / skilled eyes to catch these simple errors, what are the chances they would catch errors in source code too?

How many of the studies are doing anything statistically interesting though [ie new/different in the field of statistics]. In most cases it should be possible to have an analysis program, perhaps specialised to the field, in which you simply load the data attached to the study and look at however many regressions or time series or whatever you want - they're standard tools, surely you can download the data and put it in a GUI and look at the same graph as is in the paper in 2 or 3 clicks?? If not why not?

Re: An alarming number of scientific papers contain Excel errors

#28

Post by arethuza »

7952 wrote: post #19
ramblenode wrote: post #11

Earlier quoted context omitted.

I would counter with "use the right tool for the job." Excel is great at churning out fast and dirty estimates for low impact work. The problem is when it's used for large, complex, or important problems because these are just not Excel's domain--something obvious when looking at the kinds of features and bug fixes MS has prioritized over the years.

It is damning that you cannot use a basic computer tool for anything that is large, complex, or important. Isn't that exactly what computers are for? Call me idealistic but I want a spreadhseet that can handle milions of rows. Maybe add a "strict" mode to keep people out of trouble.

PowerPivot in Excel does allow you to have millions of rows in tables - mind you that isn't in the standard spreadsheet functionality, but it is there.

Re: An alarming number of scientific papers contain Excel errors

#29

Post by triplesec »

Anyone doing serious statistics uses SPSS or R, or similar stats programs. If not, you deserve all the bad data you get. Using Excel for that is akin to using a point and shoot camera for a fashion photoshoot, or a crossover car offroad in Death Valley.

Re: An alarming number of scientific papers contain Excel errors

#30

Post by vanderZwan »

Highly relevant: Felienne[0] Hermans' compsci research on spreadsheets out there in the wild, and how to develop software engineering tools to make them better:

https://www.youtube.com/watch?v=2Cdgew5zvI4

http://www.felienne.com/archives/tag/spreadsheets

[0] pronounced Fay-lee-nuh

Post reply on HN