Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An alarming number of scientific papers contain Excel errors

washingtonpost.com

Post reply on HN
Print view View original

31–40 of 74 posts

Re: An alarming number of scientific papers contain Excel errors

#31

Post by auxym »

ramblenode wrote: post #3

MS Excel is absolutely unfit for most scientific and engineering problems. The spreadsheet GUI, lack of good version tracking/history, and eagerness to coerce data types and "correct" values makes it easy to introduce errors that will go unrecognized and propagated through calculations. Unfortunately this story just keeps repeating itself. But all of this is just a secondary concern to Excel's real trouble: it's hist…

I can assure you that most "traditional" engineering (thinking mostly mechanical and civil) calculations, for things you use daily, were done in excel. Anything that would be a pit too much of pain to manually bang out on a calculator, but that doesn't really require complex logic (looping etc)? Excel.

Most engineers are more similar to business people in that regards, they see traditional programming languages as too complicated. We learned MATLAB in university, two classes, basic intro to programming and then numerical methods. Many people had to retake the first class at least once.

I'd love to bang out my calculations in IPython Notebooks, but the most important requirement of engineering calculations is that they can be documented and understood by peers. Since none of my peers are interested in learning python, it's useless. Excel is the lowest common denominator, everyone gets it.

I once read this interesting contrast on HN and it stuck with me:

"Traditional programming languages show the program but hide the data. Spreadsheets hide the program but show the data".

Re: An alarming number of scientific papers contain Excel errors

#33

Post by Noseshine »

pbhjpbhj wrote: post #26
Noseshine wrote: post #17

Earlier quoted context omitted.

I read all my WP articles via right-click "Open link in incognito window". (OT now:) If anyone thinks that's cheating them out of money - the choices are not "read for free" and "read for the appropriate price", the choices are "read for free" or "don't read". The reason I read is to procrastinate, so the value I get out of it is actually negative (same with HN...). Even their occasionally excellent articles (like th…

>The reason I read is to procrastinate, so the value I get out of it is actually negative (same with HN...). // Entertainment has value doesn't it? It's not a simple financial value like the simplistic use of opportunity cost as being the price you could bill those hours at, but it's a useful and functional part of being human. I don't have a problem with you reading something people broadcast to the public internet…

    > Entertainment has value doesn't it?
If it's procrastination the net value is negative. You may put any value you like on the "entertainment" - but what it displaces has higher value.

If I delay my work (which I do, even now) the overall value of writing comments on HN or reading a WP article that talks about issues that don't directly concern me and that I cannot do anything about is negative, even to myself (and don't try to argue they may concern me indirectly because, well, everything does).

It's like being addicted to drugs: Sure you can argue if the drugs (and let's assume those especially crazy and destructive ones) had no value to the person taking them they would not take them, but a more appropriate model than high school economics would be the neuroscience of addiction. But even if you decide to stick to using an economic model you would have to take a very narrow view - like picking exactly the period where a stock was rising to show how great a pick that company is - to argue the person gets a positive value from taking those drugs.

Re: An alarming number of scientific papers contain Excel errors

#34

Post by pbhjpbhj »

Noseshine wrote: post #33
pbhjpbhj wrote: post #26

Earlier quoted context omitted.

>The reason I read is to procrastinate, so the value I get out of it is actually negative (same with HN...). // Entertainment has value doesn't it? It's not a simple financial value like the simplistic use of opportunity cost as being the price you could bill those hours at, but it's a useful and functional part of being human. I don't have a problem with you reading something people broadcast to the public internet…

> Entertainment has value doesn't it? If it's procrastination the net value is negative. You may put any value you like on the "entertainment" - but what it displaces has higher value. If I delay my work (which I do, even now) the overall value of writing comments on HN or reading a WP article that talks about issues that don't directly concern me and that I cannot do anything about is negative, even to myself (and d…

>You may put any value you like on the "entertainment" - but what it displaces has higher value. //

Disagree. This conversation has a value, I [likely] can't derive anything financial from it, one might term it "entertaining" even [that wasn't supposed to sound quite so denigrating!]. The value is difficult to define, but it doesn't remove value from my life IMO. It possibly takes some time with which I can argue for opportunity cost, but I see the conversation as a generally positive thing.

I'm not sure if a mere conversation can be equated so easily to the value positions involved in drug addiction. However, I would say that it's a mixed bag. Some aspects that come out of drug addiction can have positive value - I'm thinking the progression of the arts: some great works of literature, paintings, dramatic performances, appear to have at least some relationship to the artists drug use [and in some cases addiction, it's hard to know where the divide is].

>"This is no more true than to say that Van Gogh was only Van Gogh because of his inner turmoil or than Jean-Michel Basquiat needed heroin to draw or paint. But it is also worth remembering that it killed them both." (http://www.worldcrunch.com/culture-society/under-the-influen...)

Similar ground with a greater focus on musicians - http://blogs.scientificamerican.com/mind-guest-blog/creativi....

Re: An alarming number of scientific papers contain Excel errors

#35

Post by JumpCrisscross »

SNvD7vEJ wrote: post #13

Why is the auto-convert 'features' in Excel not opt-in? When Excel encounters the first cell in a new sheet that it thinks should be auto-converted, why does it not ask if that is desirable for that sheet? Like: "Do you want Excel to interpret and auto-convert all strings with format into the type in this sheet?" At least for conversions where the original data is lost.

I don't think Excel ever destroys the underlying data. It simply layers formatting over it. One can override by specifying custom formatting, or opt out by selecting all cells and setting their format to text.

Re: An alarming number of scientific papers contain Excel errors

#36

Post by Noseshine »

pbhjpbhj wrote: post #34
Noseshine wrote: post #33

Earlier quoted context omitted.

> Entertainment has value doesn't it? If it's procrastination the net value is negative. You may put any value you like on the "entertainment" - but what it displaces has higher value. If I delay my work (which I do, even now) the overall value of writing comments on HN or reading a WP article that talks about issues that don't directly concern me and that I cannot do anything about is negative, even to myself (and d…

>You may put any value you like on the "entertainment" - but what it displaces has higher value. // Disagree. This conversation has a value, I [likely] can't derive anything financial from it, one might term it "entertaining" even [that wasn't supposed to sound quite so denigrating!]. The value is difficult to define, but it doesn't remove value from my life IMO. It possibly takes some time with which I can argue for…

You can disagree all you like - on what basis??? How do YOU know what I'm doing and what my time is worth? This discussion certainly has zero value - more like a negative value to have to encounter so annoying. Ridiculous!

    > This conversation has a value
I covered that!!! Do you actually READ the comments you respond to? I mean, without the filter that removes the things that don't fit your narrative?

Re: An alarming number of scientific papers contain Excel errors

#37

Post by coldtea »

10198231001 wrote: post #14
krapht wrote: post #5

Earlier quoted context omitted.

I'll just quote Soustrup here: "There are only two kinds of languages: the ones people complain about and the ones nobody uses." In particular you see the spreadsheet GUI as a downfall, but I think it is a great enabler that has allowed millions of non-programmers to create useful programs.

If the programs contain errors, they are not useful. They appear to be useful to someone who just wants to see some activity instead of actual work.

All programs contain errors, and almost all output of calculations is approximate.

The question is if for most of the use cases the margin of error is just fine...

Re: An alarming number of scientific papers contain Excel errors

#38

Post by ethan_g »

Rapzid wrote: post #22
ramblenode wrote: post #11

Earlier quoted context omitted.

I would counter with "use the right tool for the job." Excel is great at churning out fast and dirty estimates for low impact work. The problem is when it's used for large, complex, or important problems because these are just not Excel's domain--something obvious when looking at the kinds of features and bug fixes MS has prioritized over the years.

Excel is used HEAVILY at all levels of business for large, complex, and important things. I'm curious what this right tool is that they should be using instead.

Java, for example.

Yes, it's true Excel is used for a lot of big important things, but I'd agree with GP it's not very good at them. Moreover, I think most people who use Excel for big important things, would agree it's not good for it--they get stuck in that situation when a small spreadsheet grows big over the years. Lack of version control, limited modularity, and no ability to unit test make large spreadsheets error-prone, and Excel tends to barf on you when you push it too hard (seems to be a number of crash bugs and performance bugs that never get fixed).

Excel is incredibly productive for quick analysis and making charts. But it's really not a development platform and it's really not for heavy lifting.

Re: An alarming number of scientific papers contain Excel errors

#39

Post by ethan_g »

cm2187 wrote: post #7
krapht wrote: post #5

Earlier quoted context omitted.

I'll just quote Soustrup here: "There are only two kinds of languages: the ones people complain about and the ones nobody uses." In particular you see the spreadsheet GUI as a downfall, but I think it is a great enabler that has allowed millions of non-programmers to create useful programs.

In fact a lot could be said of the millions of ways to shoot oneself in the foot with programming languages. Excel being visual (the formula is evaluated/tested immediatly after being entered), I'd argue a diligent user will make less mistakes with Excel than while coding.

This is true for small spreadsheets, and one reason why I love Excel for quick analyses. Debugging is nearly instant because you see the results as you go along. But it doesn't scale very well.

--If you want to re-use a formula, you basically have to copy-paste it. Generally there isn't much modularity. Programmers are well aware of the dangers of this.

--It's also not a friendly platform for writing regression tests (possible with VB, but who does that?).

--The two-dimensional nature of the spreadsheets and lack of loops means you often need to do hacky stuff to emulate multidimensional arrays or loops. Easy to make mistakes here.

Re: An alarming number of scientific papers contain Excel errors

#40

Post by ethan_g »

JumpCrisscross wrote: post #35
SNvD7vEJ wrote: post #13

Why is the auto-convert 'features' in Excel not opt-in? When Excel encounters the first cell in a new sheet that it thinks should be auto-converted, why does it not ask if that is desirable for that sheet? Like: "Do you want Excel to interpret and auto-convert all strings with format into the type in this sheet?" At least for conversions where the original data is lost.

I don't think Excel ever destroys the underlying data. It simply layers formatting over it. One can override by specifying custom formatting, or opt out by selecting all cells and setting their format to text.

Commas are an exception to that. If you paste in "10,100", it will be interpreted as a number, and you lose the comma.
Post reply on HN