Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Is Microsoft Excel an Adequate Statistics Package?

practicalstats.com

Post reply on HN
Print view View original

71–80 of 108 posts

Re: Is Microsoft Excel an Adequate Statistics Package?

#71

Post by shostack »

throw_away_777 wrote: post #14
twunde wrote: post #10

I disagree. Where I work, we have one person that does all the hedging, price calculations and other financial modeling and he only uses Excel. The only practical downsides to his use of Excel it's that he doesn't have direct database access so we generally need to create the initial reports to give him the data and secondly Excel can only hold a little over 2M rows.

How do you verify that his calculations are correct?

Why do you think this can't be verified? You can see the formulas in cells, or you can see the macro code. Further, you could auto import older data inputs and outputs via data connections in Excel and set up some automated checks to flag if something is off.

Is there a use case where you think this wouldn't be possible?

Re: Is Microsoft Excel an Adequate Statistics Package?

#72

Post by goatlover »

blahi wrote: post #44
lordnacho wrote: post #34

Earlier quoted context omitted.

No way, not data science. How are you going to load in millions, possibly billions of individual numbers? With finance there's a lot of things that seem like good fits, but only if you keep them small. Something like a personal budget is fine, where everything fits on a screen.

I have 2.4 billion data points loaded in Excel at the moment. Not that it matters. "Data science" is not something that magically kicks in after you go beyond some "big data" threshold. Excel is heavily used in managerial science type of positions and I can assure you those are rather heavy on the "data science" workflows.

Why on Earth would you use Excel for that many data points?

Re: Is Microsoft Excel an Adequate Statistics Package?

#73

Post by infinite8s »

learningman wrote: post #68
ramblenode wrote: post #47

A point I have not seen mentioned is that Excel encourages bad practices for data visualization. No other statistics or data analysis software I am aware of gives you the option of 3D-ifying a 2D plot. This adds negative value to the plot for anyone who is actually interested in data over eye candy. Simple example is a pie chart. Give it depth and it becomes much more difficult to reason about, and one's reasoning co…

Agreed for the most part, although there are rare valid uses for 3d graphs. I think most contemporary data vis people would say that Excel makes it possible to make pie charts at all means it may encourage bad practices.

What are those rare valid uses for 3D graphs?

Re: Is Microsoft Excel an Adequate Statistics Package?

#74

Post by peatmoss »

dandermotj wrote: post #62
tangue wrote: post #3

Considering Microsoft recent involvment with R I have the crazy hope that future versions of Excel will be shipped with R as a scripting language.

So is Hadley Wickham... https://twitter.com/hadleywickham/status/748392441154248704

Well, later in that thread he did write:

"I am worried by the number of people that couldn’t tell that this was sarcasm"

Re: Is Microsoft Excel an Adequate Statistics Package?

#75

Post by learningman »

infinite8s wrote: post #73
learningman wrote: post #68

Earlier quoted context omitted.

Agreed for the most part, although there are rare valid uses for 3d graphs. I think most contemporary data vis people would say that Excel makes it possible to make pie charts at all means it may encourage bad practices.

What are those rare valid uses for 3D graphs?

I can think of one-- we had a 96-element datatable (12x8) and limited space in our biochemical journal article. We were looking to graphically show differences in orders of magnitude to help explain the assay. A 3d column chart fit the bill because we could show lots of data in a small space, and readers could quickly see the range of values.

Re: Is Microsoft Excel an Adequate Statistics Package?

#76

Post by ramblenode »

learningman wrote: post #68
ramblenode wrote: post #47

A point I have not seen mentioned is that Excel encourages bad practices for data visualization. No other statistics or data analysis software I am aware of gives you the option of 3D-ifying a 2D plot. This adds negative value to the plot for anyone who is actually interested in data over eye candy. Simple example is a pie chart. Give it depth and it becomes much more difficult to reason about, and one's reasoning co…

Agreed for the most part, although there are rare valid uses for 3d graphs. I think most contemporary data vis people would say that Excel makes it possible to make pie charts at all means it may encourage bad practices.

I was hesitant about the pie chart example but it was the simplest I could think of. I'm glad someone brought this up. ;)

Re: Is Microsoft Excel an Adequate Statistics Package?

#77

Post by throwaway_exer »

Delmania wrote: post #33
nn3 wrote: post #13

Earlier quoted context omitted.

The article disagrees with you """ Solution #2: Alternatives to Excel Yalta (ref 1) states that p-values [inverse probability distributions] reported by the free OpenOffice’s Calc spreadsheet and the open-source Gnumeric spreadsheet do not have the same numerical problems as does Excel - their programmers used accurate algorithms.""" It is not surprising because with an open source program everyone who can program ca…

> It is not surprising because with an open source program everyone who can program can fix such issues, while with Microsoft you are at the mercy of the likely overworked Excel team. Yep, that's the theory behind open source applications. The reality is that in a company, people will prefer Excel because Microsoft is a point of contact that can work with, blame, or yell at to fix because you're paying them. With Ope…

> The reality is that in a company, people will prefer Excel because Microsoft is a point of contact that can work with, blame, or yell at to fix because you're paying them.

Unless you are a company with more than 50,000 employees, Microsoft is very unlikely to fix anything for you, in any product.

They do poll large companies for their features roadmap, so it pays to be a megacorp.

Re: Is Microsoft Excel an Adequate Statistics Package?

#78

Post by throw_away_777 »

shostack wrote: post #71
throw_away_777 wrote: post #14

Earlier quoted context omitted.

How do you verify that his calculations are correct?

Why do you think this can't be verified? You can see the formulas in cells, or you can see the macro code. Further, you could auto import older data inputs and outputs via data connections in Excel and set up some automated checks to flag if something is off. Is there a use case where you think this wouldn't be possible?

You can verify it sure, but it is much harder to verify excel equations than code. There is no github for Excel (besides saving with different names). You generally don't see the equations when you look at Excel - this is one of the reasons Excel is so error prone.

Re: Is Microsoft Excel an Adequate Statistics Package?

#79

Post by shostack »

throw_away_777 wrote: post #78
shostack wrote: post #71

Earlier quoted context omitted.

Why do you think this can't be verified? You can see the formulas in cells, or you can see the macro code. Further, you could auto import older data inputs and outputs via data connections in Excel and set up some automated checks to flag if something is off. Is there a use case where you think this wouldn't be possible?

You can verify it sure, but it is much harder to verify excel equations than code. There is no github for Excel (besides saving with different names). You generally don't see the equations when you look at Excel - this is one of the reasons Excel is so error prone.

Fully agree. My point was that it can indeed be verified though in many circumstances.

That said, I think there's always a balance between a quick down and dirty business solution that gets the job done, vs. something fully engineered. Additionally, it is much easier for business users to shoulder more of the workload while letting people with programming knowledge focus on other tasks.

Re: Is Microsoft Excel an Adequate Statistics Package?

#80

Post by viraptor »

princeb wrote: post #42

there are a few unique issues i face with excel beyond the stats package. 1. if you shift cells (ctrl c, ctrl v) around, delete or insert rows, you may mess up existing cell references in formulas without realizing it. your vlookups, hlookups will not change your column numbers just because you did. your vba code will not change your A1 cell references. things will blow up here in spectacular fashion. 2. if you have…

> your vlookups, hlookups will not change your column numbers

If you're not using tables yet, you should. They solve that problem easily, and make addressing more explicit (relative to the table name rather than the sheet)

Post reply on HN