Live data from Hacker News

Is Microsoft Excel an Adequate Statistics Package?

practicalstats.com

91–100 of 108 posts

Re: Is Microsoft Excel an Adequate Statistics Package?

#91
post #10

I disagree. Where I work, we have one person that does all the hedging, price calculations and other financial modeling and he only uses Excel. The only practical downsides to his use of Excel it's that he doesn't have direct database access so we generally need to create the initial reports to give him the data and secondly Excel can only hold a little over 2M rows.

Excel can handle millions of rows if he brings the data into PowerPivot.

Note: I'm a PM on Excel

Re: Is Microsoft Excel an Adequate Statistics Package?

#92
post #44

Earlier quoted context omitted.

I have 2.4 billion data points loaded in Excel at the moment. Not that it matters. "Data science" is not something that magically kicks in after you go beyond some "big data" threshold. Excel is heavily used in managerial science type of positions and I can assure you those are rather heavy on the "data science" workflows.

Why on Earth would you use Excel for that many data points?

Quik self-service OLAP. Blazing fast slice & dice and multidimensional expressions (cube formulas). I would need a corporate BI solution to get that and it won't be self-service.

Nothing beats Excel + Power BI in reporting.

Re: Is Microsoft Excel an Adequate Statistics Package?

#93
Never, there are better alternatives and especially FOSS. For windows, PSPP is there to make your life easier. If you are a spreadsheet guy Libreoffice will make you never turn back to excel and can be programmed in StarBasic. R is amazing, but if you do not want to invest Scilab and Octave are there. if you are the freeware proprietary guy Google Sheets, WPS Office or FreeOffice are better alternatives than Excel. And if you have a good PC, why not give the FOSS giac/xcas a try, you may find your sanity.

Re: Is Microsoft Excel an Adequate Statistics Package?

#94
post #13

Earlier quoted context omitted.

The article disagrees with you """ Solution #2: Alternatives to Excel Yalta (ref 1) states that p-values [inverse probability distributions] reported by the free OpenOffice’s Calc spreadsheet and the open-source Gnumeric spreadsheet do not have the same numerical problems as does Excel - their programmers used accurate algorithms.""" It is not surprising because with an open source program everyone who can program ca…

> It is not surprising because with an open source program everyone who can program can fix such issues, while with Microsoft you are at the mercy of the likely overworked Excel team. Yep, that's the theory behind open source applications. The reality is that in a company, people will prefer Excel because Microsoft is a point of contact that can work with, blame, or yell at to fix because you're paying them. With Ope…

In the rather large Australian company I work for currently my manager asked himself out loud "how many tines did we ask Microsoft for support with Office"?

The answer was - "never". They wouldn't have listened and so it was not only pointless, but it was actively frowned upon!

You can purchase a rather less expensive support contract with Collabora. You don't have to build or fix the software yourself.

Re: Is Microsoft Excel an Adequate Statistics Package?

#95

Earlier quoted context omitted.

>Where I work, we have one person that does all the hedging, price calculations and other financial modeling and he only uses Excel. We used to buy in signals from a company who did their work in Excel. I wrote some scripts to export the data and recalculate it in Python. Almost every month I found errors in their reports and had to ask them to fix it. So I recommend you fight HARD to get someone to reproduce his wor…

Your conclusion is that it was excel causing these errors and you are implying similar errors would not be made in Python. I think it is more because of your experience as a developer why you were able to spot and correct errors.

Sure, you can make errors in Python. But you can also write tests to validate the data in Python. And there is a culture of doing so in Python.

>I think it is more because of your experience as a developer why you were able to spot and correct errors.

I don't feel like I spotted errors. I wrote a script to validate the data and the script told me if there were errors (there were).

Re: Is Microsoft Excel an Adequate Statistics Package?

#96
post #87
post #53

Earlier quoted context omitted.

When you're working with people who are good with Excel, they will notice. If you're only ok with excel and just use vlookups, then you're probably going to have problems, especially since those people don't tend to keep a versioned history of their files. Really the main difference is that version control has become an ingrained habit in software development whereas with Excel it's rare

Ah, a True Scotsman is good at Excel. I understand now.

[deleted]

Re: Is Microsoft Excel an Adequate Statistics Package?

#97
post #64
post #51

Earlier quoted context omitted.

It's a misconception that work in Excel can't be documented or reviewed. We have the seed data he uses and the resulting Excel files (with the formulas he used still in them). This is the equivalent to version control for programming. The results are also reviewed against 1-2 appropriate sources (invoices, historical data, multiple other reports built by IT) for accuracy. To your very legitimate point that he's the o…

> if he left, long-term we would probably need 3-4 people to take over his work. I hope you're paying him really well .

Or they could drop him to sort out the operational risk aspects.

Re: Is Microsoft Excel an Adequate Statistics Package?

#98
post #10

I disagree. Where I work, we have one person that does all the hedging, price calculations and other financial modeling and he only uses Excel. The only practical downsides to his use of Excel it's that he doesn't have direct database access so we generally need to create the initial reports to give him the data and secondly Excel can only hold a little over 2M rows.

I'm pretty sure there is an ODBC plug-in for Excel

I understood it as that the person didn't have database access, not Excel.

Re: Is Microsoft Excel an Adequate Statistics Package?

#99
post #69

A couple notes from a heavy Excel and heavy R (and heavy several other similar products) user... factories run on excel...terrifyingly so. I know semiconductor semiconductor fabs that basically run on queries in a many sheeted workbooks. The reason is because not everyone codes, especially not factory foreman, technicians, etc. They don't need a continuously running app (well they do but they don't know it) they just…

> factories run on excel...terrifyingly so Chemical plants too. But that does not mean there is no code. An Excel document can be programmed with VBA to make connections to network resources, read/write files, send emails, etc. I might not be the most effective tool for every task but it's everywhere and allows workers to automate work without making a request to hire a developer or buy new software. Someone can grad…

absolutely agreed. I mean terrifying more in the sense of the lack of controls that I have typically seen in place and the way 'pull request equivalents' are handled and modifications are made.

Re: Is Microsoft Excel an Adequate Statistics Package?

#100
post #10

I disagree. Where I work, we have one person that does all the hedging, price calculations and other financial modeling and he only uses Excel. The only practical downsides to his use of Excel it's that he doesn't have direct database access so we generally need to create the initial reports to give him the data and secondly Excel can only hold a little over 2M rows.

Excel can handle millions of rows if he brings the data into PowerPivot. Note: I'm a PM on Excel

Random point but Excel sure could use a CONTAINS function for substring matches.

I was looking for something like this yesterday and the solutions I found were quite ugly.

Ended up having to use DSUM/DCOUNT instead which is still inelegant when one has to to multiple lookups with slightly varying parameters.

Post reply on HN