Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ditching Excel for Python in a legacy industry

amypeniston.com

Post reply on HN
Print view View original

241–250 of 289 posts

Re: Ditching Excel for Python in a legacy industry

#241

Post by ricklamers »

meztez wrote: post #50

I'm pushing for our actuarial team to transition to more R + Git. After 3 years of preaching, most of the actuaries now use RStudio + git as their primary work tool. It is happening. What we did : 1) Provide documentation on everything from install to using internal R libraries for ETL. 2) Provide mostly problem free, always updated VMs with RStudio Server/ Shiny Server. 3) Establish an hotline channel for instant he…

Would love to have a chat about how you’re making R + git more accessible. How do I best reach out?

Re: Ditching Excel for Python in a legacy industry

#242

Post by ryukafalz »

mark-r wrote: post #122
mint2 wrote: post #105

Earlier quoted context omitted.

Why do actuaries refer to workstation/desktop computers with more than 16 cores as “super computers” it’s embarrassing but sometimes I give in an say “the super computer” because I’m in a hurry and they’ll give me a blank stare if I call it a workstation or anything like that.

They really are supercomputers though. Do you know how much faster a modern PC is compared to say a Cray-1? Especially if it has a decent graphics card.

Yeah but at this point my phone is comparable. If being faster than a Cray-1 qualifies something as a supercomputer, the definition is meaningless now.

Re: Ditching Excel for Python in a legacy industry

#243

Post by etothepii »

keithalewis wrote: post #213

This reminds me of Bill Gates' comment about secretaries writing VBA. Thankfully, that didn't happen. My friend rides herd on all the python code written at BigBank. He said they now have over 1MM python scripts, "Most of them written by people who had no business writing code in the first place." Python certainly lowers the barrier to writing code, but I seriously doubt many Excel power users will jump on the Python…

This is a great point but companies are using tools built by amateurs and just don't know it. I think of a lot of business users as the fine artisans you might have come and paint a fresco at the Sistine wall. Sure, they might be the best painter in the world, but if they are painting on a poorly constructed building there will be little-to-no long term sustainable value.

Re: Ditching Excel for Python in a legacy industry

#244

Post by zentiggr »

pjmlp wrote: post #226
etothepii wrote: post #199

Earlier quoted context omitted.

If the code was stored in source control with a separation between dev and production the intern might be able to raise such a Pull Request but it would never get approved without oversight from someone more senior. This Code Review process is almost completely impossible in Excel.

That is a big if. Office has source control management via SharePoint integration.

... and then you have two problems. I have never seen an environment converted to SharePoint that didn't suffer badly from the conversion.

And the Excel vs source control issue isn't history, it's "go ahead and try to diff between two versions of an Excel sheet" vs "diff two versions of that source file".

Unless I've never heard of the tool that can digest two Excel sheets and tell you what formulas differ, or cells. Please correct me, anyone who knows of one.

Re: Ditching Excel for Python in a legacy industry

#245

Post by etothepii »

rvba wrote: post #224
mslate wrote: post #29

Earlier quoted context omitted.

From an actuary's perspective: is Google Sheets ever entertained as an Excel alternative?

Uploading propertiary data to a cloud is a big no.

Companies are more worried about this than they probably should be.

The average enterprise network is nothing like as secure as people behave like it is.

Where do you think your email is hosted? With few exceptions I'd expect its provided by a cloud provider these days.

Re: Ditching Excel for Python in a legacy industry

#246

Post by etothepii »

pjmlp wrote: post #226
etothepii wrote: post #199

Earlier quoted context omitted.

If the code was stored in source control with a separation between dev and production the intern might be able to raise such a Pull Request but it would never get approved without oversight from someone more senior. This Code Review process is almost completely impossible in Excel.

That is a big if. Office has source control management via SharePoint integration.

An excel tool that showed you formula differences sounds like a cracking idea for a startup.

Re: Ditching Excel for Python in a legacy industry

#247

Post by etothepii »

rvba wrote: post #230

I dont claim that Excel is the best tool or that it should be used in reinsurance, but it seems that the author does not know how to use it properly. It sounds more as if they knew Python so they use Python. If author would know Java they would write that Java is better. For example combining data with logic does not need to happen in Excel. Someone who does this might make the same bad practice in Python too. Also t…

There are some good criticisms here. However, the OP is getting at the fact that at least the tools exist for code review, source control and automated testing in Python.

Re: Ditching Excel for Python in a legacy industry

#248

Post by z3t4 »

yold__ wrote: post #87
stochastastic wrote: post #70

Earlier quoted context omitted.

Hey fellow reinsurance actuary! I totally agree that Excel has its place in modeling, especially one-offs, and your criticisms make sense. That said, we have been moving a lot of our calculations to Python. We have had way too many rickety tools to move files or send emails (“first you open this spreadsheet and click this button, then you open this spreadsheet and click this button, then...”), and way too many versio…

One reason why Python is so successful is that it places very nicely with C code. Many of Python's libraries are thin wrappers around native DLLs. For example, numpy is a wrapper around a BLAS DLL (e.g. Intel MKL). Pipenv manages the python side of things, but don't exert control over the system DLLs (like Docker does). Anaconda gets very close to what Docker does (by managing DLLs). Have not used poetry, so can't co…

In Nodejs native libraries are a PITA. The compatibility API breaks on a schedule every 6 month, dependencies get updated by OS distros sometimes breaking stuff, then you rely on the OS being able to compile the library. I hope Python has better native interop.

Re: Ditching Excel for Python in a legacy industry

#249

Post by pjmlp »

zentiggr wrote: post #244
pjmlp wrote: post #226

Earlier quoted context omitted.

That is a big if. Office has source control management via SharePoint integration.

... and then you have two problems. I have never seen an environment converted to SharePoint that didn't suffer badly from the conversion. And the Excel vs source control issue isn't history, it's "go ahead and try to diff between two versions of an Excel sheet" vs "diff two versions of that source file". Unless I've never heard of the tool that can digest two Excel sheets and tell you what formulas differ, or cells.…

https://support.microsoft.com/en-us/office/overview-of-sprea...

Re: Ditching Excel for Python in a legacy industry

#250

Post by zentiggr »

listenallyall wrote: post #110
bash-j wrote: post #19

Earlier quoted context omitted.

This has frustrated me as a python user for the last 7 years, working as the only python user in business environments dominated by Excel. People will say things like, "if you leave, who can support this report you made in python?" Well I say, who can support the bloated 40mb spreadsheet that would take forever to unpick and figure out how to update with new data? No one can, because I've seen people would rather reb…

> You can't even count on the data being in the same place over the 52 weeks, since they would have added and removed data points over time. > I can just save it as a .py file and run it as a scheduled task on a virtual computer forever. This is rather naive and short-sighted. Do you think the spreadsheet guy is moving data points around because s/he's bored at work and is screwing around with no purpose? No, the bus…

It sounds like you missed the point of GP's post.

He was talking about avoiding manual weekly data copy-paste errors by writing code to do it in a predictable format.

I think you assumed that they meant the code would never have to be changed again, when they were actually talking about being able to standardize the data update process.

I don't think they said anything about never having to change the code, just that running a software process saves each user from manually pasting data every week.

Post reply on HN