In R, you can use accepted methods of software development and engineering, such as version control, automated testing, CI/CD to catch mistakes and prevent them from causing harm. In Excel? Not really.
Do you think the average finance department employee is going to do that?
Especially those departments should do so and learn to do it. Financial departments are always the ones complaining about the cost of mistakes, compliance risks (of the monetary kind) and doing due diligence. Preventing the production of wrong figures in financial excel sheets ticks all those boxes. They should bloody well be doing it, but I'm also not holding my breath ;)
The tooling is massively better than Python to say the least. You have a single environment (RStudio) that does most things really well, an excellent package manager, and integrated graphing capabilities and environment variables observation. In comparison Jupyter notebooks feel very limited and clumsy. And Base-R is way more tuned towards data analysis than base Python is, so you end up importing many libraries in P…
The tidyverse packages are pretty great that is true, in particular dplyr was useful. I don't want to sound like I'm just someone against R for the sake of it. There were things I liked, but I just walked away from the project uncertain what the benefit was in using R. There's a solid chance i'm wrong.
The advantage is you get tables, numbers and plots based on solids maths instead of implementing your own understanding of useful analyses. Depending on your skill and interests in implementing your own designs for stats this might be a plus or a minus. For me, I don’t want to spend my time redoing the tricky and subtle stuff, but I can learn the odd syntax and weirdly functional nature of it pretty reliably. If someone does a Golang version of CRAN I wouldn’t think of using R again but for now R wins the stats battle. And much of the community advice and so on about how to do things, outside of the base R vs tidyverse flame rift, are from informed statisticians not autodidact Python folks that think everything is easy. But if you do understand he maths solidly behind what you want, then R might not be the best tool for your use.
Also -Open source is better for serious work -Excel is notoriously buggy and shouldn’t be used for anything important, yet it is and the costs are enormous. Many articles on this.
I highly recommend anybody getting into R to skip the base language (which indeed is ancient and full of gotchas) and go straight for the Tidyverse[1]. You can always go back in and learn how to do things the old way later. Over the last decade, the R community has largely standardized around tools like dplyr, ggplot, tibble, purrr, and so on that make doing data science work way easier to reason about. Much more erg…
i would recommend getting comfortable with doing stuff with base R, then trying tidyverse. Starting with dplyr might get you results quick, but its "special evaluation" actively confuses your understanding of how the base language actually works (speaking from experience with an R course and subsequently helping other confused folks) Consider this example: # base R starwars[starwars$height % filter( height (Source: h…
My experience is that this weird evaluation order stuff is only confusing for students with a lot of programming experience who already expect nice lexical scope. For those coming in from Excel, the tidyverse conventions are no problem and are in fact easier than all the pedantic quoting you have to do in something like Pandas. It only gets confusing when you want to write new tidyverse functions, and even then, base R isn’t any simpler: the confusing evaluation order is built into R itself at the deepest level.
finance people who learn to program will outperform those who don't.
I’m a hobbyist programmer who has worked in various finance jobs and is now working in investment banking. I would love to use programming in finance but I have to say it’s come in handy very little.
Well, prolly not in IBD. But in Sales and Trading or at a lot of quant funds it is very common.
Excel doesn't alter data without being asked either. What it does do by default is try to change how the data is displayed. If the column is properly set to the correct data type the data will be shown correctly because Excel doesn't change the actual data. Column types can be set when csv files are imported. The problem described in the article isn't an Excel issue. It is an issue of the geneticist failure to learn…
the only way to prevent Excel from converting date-like number values into 'real dates' is to format the number values as text. or escape each and every number value so this doesn't occur. this is not just a workaround - it's recommended by Microsoft. because you literally cannot turn this functionality off. how is that not an "Excel issue"?
That is how spreadsheets work. Try Libreoffice or Google Sheets. You will find they operate the same. Spreadsheets format all cells as a generic type where the value entered to it is gets assigned a more specific type by the spreadsheet application. If you want to insure that the value you entered is interpreted as a specific type you need to identify that to the spreadsheet application prior to entering the value.
Most ppl who use excel would benefit from Access and creating some repeatable processes. A lot of the sorting and ranking from excel could be done in linked tables and improve the repeatability
For non-programmers who do any kind of data processing, spreadsheet applications (Excel, Numbers, Google Sheets) are the closest thing to programming they have, and have a lot less overhead - just throw in the data, write a formula, you get results, no compilation or fidgeting with input and output and so on. The steps of an algorithm are reflected by cells that reference cells that reference cells. I've always thoug…
> it really is a superpower in many contexts. It's a trap because once you get comfortable in Excel you have a lot of resistance to try anything more productive than Excel. Seen that numerous times with people who work really fast with Excel yet end up very limited as to what they can actually deal with beyond simple problems.
Disagree. Wanting to automate and learning functions within functions pushed me to learn how to program in python and R
If you need to do something once, use a GUI (Excel). If you need to do something ten times, use hotkeys and shortcuts. If you need to do something a hundred times, write a script (R). I usually use the command line as the example for why writing code and scripts are better than the more intuitive and lower-learning-curve GUIs. If I want to move a file from one folder to another then I just drag it across. Easy. If I…
Also, if you need to do something (even once) and have to know how you did it three months later, write a script.
Yes, the lack of reproducibility is really Excel’s biggest weakness in data analysis. Is this 5th workbook that is a 4th generation derivation from the original data statistically valid? Can you go back to each step and verify it was valid? A script that does all processing and analysis starting from raw data can be validated and audited.
Most, but not all. That is the entire point: Excel is never safe in any configuration.
If "not all", can you provide an example of what gets converted if you specify text? Just want to know to avoid future gotchas.
I'm not sure, have to try it out but text fields can be delineated by starting with a single quote ('), that might be a special case. It might not be though, I don't have Excel open to check. Pretty much everything is left alone if you specify "text" at the import stage though. (This is using the text import wizard, or text-to-columns)