Live data from Hacker News

Ditching Excel for Python in a legacy industry

amypeniston.com

151–160 of 289 posts

Re: Ditching Excel for Python in a legacy industry

#151
post #39

Earlier quoted context omitted.

My first job was cleaning up the mess that was caused by hard coded yahoo finance urls in spreadsheets. One of them died, no one noticed and it cost the company millions of dollars in bad trades over three months.

are you suggesting programmers don't hardcode URLs in quick-and-dirty (and sometimes, even production) Python scripts/code?

I'm suggesting they have logs.

Re: Ditching Excel for Python in a legacy industry

#152
post #26

I'm a research actuary working in reinsurance. Here is why I think Python creates more problems than it solves from the standpoint of most insurance business users: 1.) Environment management. There are many solutions for managing python dependencies, my favorite is Docker + pip. Good luck getting actuaries and underwriters to write Dockerfiles etc, and good luck getting I.T. to support Docker on Windows desktops. Li…

Funny story about Excel on corporate machines. A couple of years ago the company I work for got boight by an Italian company. When we finally migrated the Windows users over to the corporate Office installs a bunch of people found that Excel wouldn't work for them. Things like sum(A1:A20) were syntax errors. After a bunch of digging i worked out that the localisation from corporate meant they suddenly had Italian fun…

> There are ways of building spreadsheets that are reliable and auditable. Then there's how 95% of people do it

Do you have any pointers to learning materials on how to do this? Would be interested in reading more on it.

Re: Ditching Excel for Python in a legacy industry

#154
This is actually about using a paid, closed-source add-on PyXLL, that integrates python into Excel, but only to the Windows version.

And costs 25 USD per month (but has a free trial).

Excel is never actually ditched.

edit: oh, that's only step 4. Step 5 is actually ditching Excel.

Re: Ditching Excel for Python in a legacy industry

#155

Earlier quoted context omitted.

Why doesn't annotating these functions with @functools.lru_cache(10000000) work?

Not sure why this is being downvoted since I don't think the OP has done a good job of showing evidence that this doesn't work for recursion. Your computer almost certainly WILL have enough space for all the stack frames necessary.

[deleted]

Re: Ditching Excel for Python in a legacy industry

#156
post #26

I'm a research actuary working in reinsurance. Here is why I think Python creates more problems than it solves from the standpoint of most insurance business users: 1.) Environment management. There are many solutions for managing python dependencies, my favorite is Docker + pip. Good luck getting actuaries and underwriters to write Dockerfiles etc, and good luck getting I.T. to support Docker on Windows desktops. Li…

Hey fellow reinsurance actuary! I totally agree that Excel has its place in modeling, especially one-offs, and your criticisms make sense. That said, we have been moving a lot of our calculations to Python. We have had way too many rickety tools to move files or send emails (“first you open this spreadsheet and click this button, then you open this spreadsheet and click this button, then...”), and way too many versio…

Much like the referenced Excel spreadsheets becoming unwieldy, so does a dev's machine [0].

0. https://xkcd.com/1987/

Re: Ditching Excel for Python in a legacy industry

#157

Earlier quoted context omitted.

> What does "A1 + SomeOtherTab:B2" mean? Excel offers the ability to name any cell or range of cells. Don't even have to search through menus or the ribbon, it's right there to the left of the formula bar. I'm well aware that the vast majority of Excel spreadsheets don't use named cells/ranges, but you can't really blame Excel for that. It couldn't be too much easier. Lots of Python programmers don't use comments or…

> Excel offers the ability to name any cell or range of cells Except almost nobody does. It's not intuitive or the way it's taught. > Lots of Python programmers don't use comments or descriptive variable names either. You have to have variable names in Python. If you want to give them shitty names, that's your bag, but unlike Excel, it's not an extra step. You also have to deal with the fact that every cell in a rang…

Interesting, my comment was solely in response to your complaint about naming convention... you've expanded your criticism quite a bit. I already anticipated your issue with naming being that few people used named cells/ranges -- again, it's not even in a menu or ribbon, it's present at all times, what else do you want? Not "the way it's taught"? Well, blame your teacher.

> You have to have variable names in Python

sure, but `(i, j, k)` isn't any more descriptive than A1 or B7 or CQ85759. `intOrderTotal` may seem better initially, until the summer intern creates `intOrderFinalTotal` (after tax) and `intOrderAllInFinalTotal` (after shipping and tax)

> every cell in a range has its own unique formula

you could use array formulas, or were you never taught those either?

> it's not the default behavior, takes extra steps,

Creating a Python virtual environment is not the default behavior, and it takes extra steps. So does using any packages beyond the standard library. So does source control. Or running Jupyter. Using classes, or type hints, or imports, are all not the "default" of one long script in a single file.

Excel isn't superior to Python, or the best tool to solve every type of problem. Excel has its place, Python has its place. But your specific little nitpicks here are a reflection of the user (who I presume is you) not on the tool itself.

Re: Ditching Excel for Python in a legacy industry

#158

Earlier quoted context omitted.

Why doesn't annotating these functions with @functools.lru_cache(10000000) work?

Not sure why this is being downvoted since I don't think the OP has done a good job of showing evidence that this doesn't work for recursion. Your computer almost certainly WILL have enough space for all the stack frames necessary.

[deleted]

Re: Ditching Excel for Python in a legacy industry

#159
post #151

Earlier quoted context omitted.

are you suggesting programmers don't hardcode URLs in quick-and-dirty (and sometimes, even production) Python scripts/code?

I'm suggesting they have logs.

logs of what? if a programmer hard-codes URLs, you really think they are following best practices elsewhere? further, how do logs help get back the millions of dollars that were lost?

Re: Ditching Excel for Python in a legacy industry

#160

Earlier quoted context omitted.

That's the thing, most Excel workbooks start out as a one-off then gradually get adapted and extended until they're load-bearing. They're often built by specialists in another dept who definitely wouldn't consider themselves programmers. Doing it 'properly' would probably mean having to spec put the problem, get a budget, maybe wait a few months for someone to look at it. And the same thing every time the requirement…

> until they're load-bearing This cannot be stressed enough. I've outlived generations of finance teams at many startups, and I've seen firsthand the masterpieces/abominations left behind in Excel. Imagine a dozen sheets with ad-hoc queried data copy/pasted from System A/B/C/D into Excel, with formulas that feed formulas that feed formulas. Sometimes columns are inputs (seasonality adjustments for monthly forecasts),…

I remember hearing about a mythical spreadsheet floating around for modelling something to do with our national grid a few years back.

It would take about 12 hours to calculate, and would error out before finishing about 30% of the time. It needed to be run once a day for something reasonably important.

I don't use Excel much these days, but I do point people to a video if they do plan on doing anything:

* [You suck at excel - Joel Spolsky](https://m.youtube.com/watch?v=0nbkaYsR94c)

Post reply on HN