Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ditching Excel for Python in a legacy industry

amypeniston.com

Post reply on HN
Print view View original

231–240 of 289 posts

Re: Ditching Excel for Python in a legacy industry

#231

Post by HALtheWise »

zinekeller wrote: post #48
mslate wrote: post #29

Earlier quoted context omitted.

From an actuary's perspective: is Google Sheets ever entertained as an Excel alternative?

Used GSuite (Google Workspace?): It's fine, it is improving but lack of shortcuts even for basic tasks (I can change the font on Word with just a keyboard, try that with Docs without a mouse). Dealbreaker is the 5 million cell (not row nor column) limit, which is even lower than Microsoft's old limits (more than 15 million cells, which was increased in 2007 to you-have-a- serious -problem-if-you-somehow-fill-this-lim…

Alt+/ is the magical shortcut that makes this easy, you can simply search a font name and hit enter.

Re: Ditching Excel for Python in a legacy industry

#232

Post by HALtheWise »

sam_bristow wrote: post #113
yold__ wrote: post #26

I'm a research actuary working in reinsurance. Here is why I think Python creates more problems than it solves from the standpoint of most insurance business users: 1.) Environment management. There are many solutions for managing python dependencies, my favorite is Docker + pip. Good luck getting actuaries and underwriters to write Dockerfiles etc, and good luck getting I.T. to support Docker on Windows desktops. Li…

Funny story about Excel on corporate machines. A couple of years ago the company I work for got boight by an Italian company. When we finally migrated the Windows users over to the corporate Office installs a bunch of people found that Excel wouldn't work for them. Things like sum(A1:A20) were syntax errors. After a bunch of digging i worked out that the localisation from corporate meant they suddenly had Italian fun…

This brings up a good point, which is that Excel supports localization, while Python just assumed you know English.

Re: Ditching Excel for Python in a legacy industry

#233

Post by sriku »

yold__ wrote: post #26

I'm a research actuary working in reinsurance. Here is why I think Python creates more problems than it solves from the standpoint of most insurance business users: 1.) Environment management. There are many solutions for managing python dependencies, my favorite is Docker + pip. Good luck getting actuaries and underwriters to write Dockerfiles etc, and good luck getting I.T. to support Docker on Windows desktops. Li…

If you have valid reasons to make the move from Excel to Python, why not consider Julia? Environment is easier to manage (Pkg.add), the language "looks like Python and walks like C", math-friendly style possible, just-ahead-of-time compilation resulting in high performance (enough to not need native code implementations), interactive development (Jupyter was named for Julia-Python-R after all), @memoize may be enough for you and Pluto gives you reactive notebooks.

Bonus - tools are also emerging to make stand alone distributables.

Disclaimer: I neither work for nor am I affiliated with Julialang. I just use it.

Re: Ditching Excel for Python in a legacy industry

#234

Post by klelatti »

everling wrote: post #211
mch82 wrote: post #49

Earlier quoted context omitted.

> problem #1: Environment management Great observation. Python environment management is getting simpler, but is off putting for people without a software background. Unclear even CS majors get enough classroom exposure to package & dependency management to utilize Python efficiently. I’m more optimistic about an on-prem deployment of Jupyter Notebooks or Sage Math Cloud as a way to hide a lot of the setup complexity…

Quant here, at my firm we've deployed a JupyterHub server which provides users with a production docker image, so that analysts and portfolio managers can perform analyses without installing python, dependencies and sql drivers locally. It is working well and spurs interest in Python across the wider org - so we let everyone use it. Similar to in OP's case, I think the real selling point of Python over Excel is advan…

That's really interesting. I've been working on a JupyterHub / JupyterLab / Python based product for the insurance industry - choosing this for all the reasons you've cited. Would be really interested if there are any points you can share e.g are you using Kubernetes and if so how have you found it?

Re: Ditching Excel for Python in a legacy industry

#235

Post by x87678r »

jbjbjbjb wrote: post #17
x87678r wrote: post #7

I do a lot of both. Excel really is great for data where there are less than say 100k rows. Its just so easy to see exactly what you're doing and what the data looks like. If you have millions of records Python really does better but I still find it frustrating to find a way to keep peeking behind the curtain. Ideally I'd have a type safe language which can embed data the way excel does. If Excel had dotnet languages…

Try ExcelDna it lets you hook up .net to Excel

Yes I use Excel DNA, it just doesn't have the same flexibility that I can email anyone a spreadsheet file containing both code and data.

Re: Ditching Excel for Python in a legacy industry

#236

Post by yufeng66 »

yold__ wrote: post #26

I'm a research actuary working in reinsurance. Here is why I think Python creates more problems than it solves from the standpoint of most insurance business users: 1.) Environment management. There are many solutions for managing python dependencies, my favorite is Docker + pip. Good luck getting actuaries and underwriters to write Dockerfiles etc, and good luck getting I.T. to support Docker on Windows desktops. Li…

I am an actuary for 20 years and people have been trying to replace Excel for at least 15 of them. But Excel is not going anywhere. I think it will be even more popular with the recent introduced lamda function. Excel formulas will Turing complete and we don’t have to use VBA anymore.

Re: Ditching Excel for Python in a legacy industry

#237

Post by alexilliamson »

hnracer wrote: post #108
klelatti wrote: post #57

Earlier quoted context omitted.

I'm an actuary with a strong interest in this area - would be very interested to hear more especially on your R vs Python experience.

I've used R (3 years) and Python (8+ years) in data science and much prefer Python, because it can do things that aren't just pure data analysis, and because pandas is so amazingly good compared to R's data matrix solutions, in my opinion. I believe that the algorithmic trading industry has gone fully into Python and away from R for these reasons.

I've been a heavy user of all 3, and pandas syntax is a nightmare compared to dplyR or data.table in R. That being said, I still use pandas because I prefer python for non-analysis.

Re: Ditching Excel for Python in a legacy industry

#238

Post by jgalt212 »

klelatti wrote: post #57
meztez wrote: post #50

I'm pushing for our actuarial team to transition to more R + Git. After 3 years of preaching, most of the actuaries now use RStudio + git as their primary work tool. It is happening. What we did : 1) Provide documentation on everything from install to using internal R libraries for ETL. 2) Provide mostly problem free, always updated VMs with RStudio Server/ Shiny Server. 3) Establish an hotline channel for instant he…

I'm an actuary with a strong interest in this area - would be very interested to hear more especially on your R vs Python experience.

R is better if your raw data is already tabular. I prefer Python if the raw data is unstructured / semi-structured. You can make the case that once Python has converted the data to tabular then move to R, but at that point I like the soup to nuts to be in one language.

Re: Ditching Excel for Python in a legacy industry

#239

Post by alexilliamson »

Jwilcoxdata wrote: post #99

I’m surprised that there aren’t more comments about utilizing R AND Python for analysis work. These two languages actually commingle fairly well, you can build in RStudio if you like that flavor and still import Python packages to use in R code. We do a significant amount of modeling and analysis on large data sets from a variety of disparate sources and utilizing several different packages have extended this out to…

Yes! RStudio is an amazing product, and you truly can have the best of both worlds by using python in it.

Re: Ditching Excel for Python in a legacy industry

#240

Post by m16ghost »

etothepii wrote: post #195
kqvamxurcagg wrote: post #27

Earlier quoted context omitted.

I've used excel and python in lots of business contexts. For most tasks involving domain experts, excel usually wins hands down. An excel spreadsheet is usually easily auditable. The visual presentation and layout lends itself to review by others. You can click and point at values. Python and other programming languages require an environment and tooling that can't be easily supported across the enterprise. It requir…

> An excel spreadsheet is usually easily auditable. What does this sentence mean? Virtually no part of excel is easily auditable, at best you can see "this file was changed by xyz at a:b:c" identifying the cells that have changed between versions wouldn't even be easy.

All versions of Excel from 2013 and onward come with a tool that lists out cell by cell differences between two spreadsheets, called Spreadsheet Compare [0].

It lists side-by-side differences in hardcoded values, formula changes, calculated value changes, and even changes in VBA code. The list of changes can then be dumped out to a text file.

Default Excel installations also include the Inquire add-in which allows you to perform the comparison within Excel itself.

[0] https://support.microsoft.com/en-us/office/compare-two-versi...

Post reply on HN