i think that part of the answer is that software engineers simply aren’t (usually) familiar with excel. It’s a remarkably complex and powerful program but if you never used it professionally you will never learn its full potential. Yes, it’s easier to do useful stuff as a beginner in excel vs a real programming language but to get really good at it takes time. You might think you know excel because you’ve used it in…
Why do we use R rather than Excel?
131–140 of 266 posts
Re: Why do we use R rather than Excel?
#132A while ago, there was a job offer in my area to automate Excel tasks using Python. The thing was posted time after time by several recruitment agencies, for almost a year. Just thinking how expensive that process must have been answers the question.
If I share a computation as a spreadsheet, people already know how to work with the GUI, which is actually quite sophisticated, even if it doesn't prevent you from screwing up. People could extend my spreadsheets, e.g., adding columns to do the same computation on multiple input sets, etc. They could easily extract the output in text format and paste it into something else. And so forth.
A Python program with GUI created the expectation that I was writing commercial quality "software," and that if it wasn't 100% intuitive, I would hand-hold each colleague, and make changes on demand, including converters to multiple file formats (often, so they could put the data back into a spreadsheet). And of course I also had to help each person install Python on their computer.
Of course I'm not a full fledged software developer, just a "scientific" programmer, and of course the lesson I learned is common knowledge: Writing and supporting real software is orders of magnitude more costly than just writing a one-off program to solve a problem, to the point where the conversion to "proper code" could be a net liability to the business. You have to assess whether it will actually add value. In one sibling comment, it sounds like the answer was yes, so I acknowledge that.
The difference was not so much the technology, but the cultural expectations associated with Excel versus "software."
Today, I do all of my work in Python, but when I share a simple computation with colleagues, I will often convert it back into Excel for them.
Re: Why do we use R rather than Excel?
#133Unless you need the frontier of statistical methods and are unable to code them yourself, I highly recommend not engaging R. While Hadley Wickham has done amazing things to make the R language actually useful, Python and Julia are better for data applications. Also, bindings to the R language exist, obviating the need to be tied into it completely. Thus, even if you need the frontier, you have it available to you.
I believe working in pandas/sklearn/jupyter ecosystem is much slower (>50%) if you are doing EDA and statistical modelling (ML) than tidyverse/tidymodels/rstudio. The exception is deep learning and adjacent fields (like computer vision).
Of course Python is better at everything else, so adding an extra languages might or might be worth the hassle.
Re: Why do we use R rather than Excel?
#134I think a fairer question today would be "Why do we use R rather than M?" https://www.howtoexcel.org/power-query/m-code/
Absolutely! For power-users of Excel, this article reads as "the reason we don't use Excel is because we don't understand how to use it". For those who aren't as familiar with Excel, all the functionality the author is describing as missing in Excel is actually built into it out of the box - as a feature called Get & Transform (or 'PowerQuery'). You can load in a CSV, manipulate it, do batch processing, and all the s…
Many people to this day aren't even aware cells can be named like proper variable names.
Re: Why do we use R rather than Excel?
#135The reason I use python scripts instead of spreadsheets for important calculations is that I can unit test the logic extensively and be sure that the code works as expected. This also makes sure I don't break stuff when adding functionalities / refactoring code.
I would never ever use Excel for something important, unless it's completely trivial (e.g., sum/average values of a column).
I assume the same applies to R vs spreadhseets, of course.
Re: Why do we use R rather than Excel?
#136If you need to do something once, use a GUI (Excel). If you need to do something ten times, use hotkeys and shortcuts. If you need to do something a hundred times, write a script (R). I usually use the command line as the example for why writing code and scripts are better than the more intuitive and lower-learning-curve GUIs. If I want to move a file from one folder to another then I just drag it across. Easy. If I…
Re: Why do we use R rather than Excel?
#137Earlier quoted context omitted.
Excel doesn't alter data without being asked either. What it does do by default is try to change how the data is displayed. If the column is properly set to the correct data type the data will be shown correctly because Excel doesn't change the actual data. Column types can be set when csv files are imported. The problem described in the article isn't an Excel issue. It is an issue of the geneticist failure to learn…
> Excel doesn't alter data without being asked either. What it does do by default is try to change how the data is displayed. This is easily falsifiable. In a "general" cell, when I enter 0002, it gets changed into 2, not just in display, but in actual content. When I change the cell type to text, it'll still be 2. Only if I enter 0002 after changing the type to text is the content kept. Similar when I want to have t…
Re: Why do we use R rather than Excel?
#138Big big reason: R doesn't alter data without being asked to, Excel does so, silently, sneakily. Anything important should not be done in Excel. https://www.theverge.com/2020/8/6/21355674/human-genes-renam... https://stackoverflow.com/questions/165042/stop-excel-from-a...
Hold on a second while I go shut down the global economy for two years so we can teach everyone finance person how to program.
Re: Why do we use R rather than Excel?
#139This week I got a bit more familiar with R while adding it as a scripting language for a data IDE I'm working on (it already supported other languages). It is a very hard language and community to get into! The documentation is very sparse. Library documentation is published as PDF (I guess?) and also very sparse. The default `print` behavior is pretty hard to understand. It's 1-indexed and it took me a while to real…
You highlight four issues: 1. Documentation is accessible via the interpreter. You can type ?funcname to get documentation or ?libname for the entry point for almost every library, or use the Help tab in RStudio, the most common interpreter. Package documentation is typically hyperlinked text and of a high quality. You can also see syndicated versions of library documentation online in HTML format. Here is for instan…
Don't take this as an attack on the language or community. I'm a huge fan of Standard ML and it's arguably in a worse state!
I was interested in supporting R in the first place because I knew of its importance (if only vaguely).
Re: Why do we use R rather than Excel?
#140Big big reason: R doesn't alter data without being asked to, Excel does so, silently, sneakily. Anything important should not be done in Excel. https://www.theverge.com/2020/8/6/21355674/human-genes-renam... https://stackoverflow.com/questions/165042/stop-excel-from-a...
> Anything important should not be done in Excel. Hold on a second while I go shut down the global economy for two years so we can teach everyone finance person how to program.
https://theconversation.com/economists-an-excel-error-and-th...
Not that this is Excel's fault, but researchers should definitely either seriously learn how to use computer stuff, or just don't.
A person can't drive on the highway without a license, the same rigor should be applied here, especially in academic circles.