An oral history of Bank Python • Hacker News Board

Re: An oral history of Bank Python

#111

Post by girvo » Thu, Nov 04, 2021, 12:44 PM UTC

kwertyoowiyop wrote: post #108

captainmuon wrote: post #18

Reminds me of what we used at the ATLAS experiment at CERN* . Python was tightly integrated with the application framework, Athena (which I just realize has the same name as JPM's Python framework!). You could use it as a job description language, and you would compose computation steps from classes you could write in C++. I think there was a separate `athena` executable that was just python with some packages pre-lo…

The Greek and Roman gods have always been a go-to for project names, LOL. We need to give some other cultures a shot!

I’ve been a fan of metals (and more broadly elements) for project code names lately. My current project is called Cobalt!

Re: An oral history of Bank Python

#112

Post by [deleted] » Thu, Nov 04, 2021, 12:45 PM UTC

lozenge wrote: post #30

tikkabhuna wrote: post #26

Earlier quoted context omitted.

I joined the BAML grad scheme 10 or so years ago. We had a presentation from one of the Quartz guys and someone asked how they’d manage upgrading the version of Python. They were using something like 2.6.5. The whole move to 3.x was a thing. The Quartz guy just flat out said they wouldn’t upgrade. Seemed crazy to a new grad back then, but now I wouldn’t want to consider it either. Thanks for your contribution! It was…

BAML(Quartz) and JPM(Athena) both had Python 3 migrations well underway as of PyCon UK 2019. It took me more than half way down the article to identify I have not worked at the same bank as the author...

[deleted]

Re: An oral history of Bank Python

#113

Post by girvo » Thu, Nov 04, 2021, 12:45 PM UTC

simonh wrote: post #48

blitzar wrote: post #27

Earlier quoted context omitted.

Or some american writes the date somewhere. edit: /s we love you american colleagues,

Working with and for Americans is why I always use ISO year-month-day format.

That, and I like how they neatly sort.

Re: An oral history of Bank Python

#114

Post by hdjjhhvvhga » Thu, Nov 04, 2021, 12:50 PM UTC

JoshTriplett wrote: post #80

seanhunter wrote: post #21

I was the person who first deployed Python at Goldman Sachs. At the time it was an "unapproved technology" but the partner in charge of my division called me and said (This is literally word for word because partners didn't call me every day so I remember) Err....hi Sean, It's Armen. Uhh.... So I heard you like python.... well if someone was to uhh.... install python on the train... they probably wouldn't be fired. O…

There are a number of workplaces where I'd have been willing to rely on "probably wouldn't be fired", but a bank is definitely not one of them. Congratulations on shipping something useful in the face of that risk and uncertainty.

> but a bank is definitely not one of them

Investment banks are basically risk-management shops. The partner made an assessment and evaluated the potential benefits as higher than risks. Note the word "probably".

Re: An oral history of Bank Python

#115

Post by Icathian » Thu, Nov 04, 2021, 12:54 PM UTC

danmur wrote: post #92

andrewshadura wrote: post #81

Earlier quoted context omitted.

> python build, dependency and deployment management is exceptionally awful in every respect, this isn't as big a pain point in other languages I'm not sure how to react to that, but these features in Python are miles ahead of what many other languages have (or actually don't have).

I think there are too many options, or not enough direction for busy people. Once you understand how it all works and pick / build the right tools it all works pretty well.

I would agree for all but deployment. I know my way around python reasonably well, but pyinstaller and friends still make me have bad days pretty regularly.

Re: An oral history of Bank Python

#116

Post by lucozade » Thu, Nov 04, 2021, 12:56 PM UTC

urban_winter wrote: post #53

BAML Quartz was conceived by a bunch of front-office quants who had not the first idea about the software needs of a big bank beyond the front office. There was an arrogant assumption that front office software is obviously the most complicated/difficult variety of software within a bank and therefore any system designed with front office requirements at the forefront would, of course, be perfect for universal use. T…

> BAML Quartz was conceived by a bunch of front-office quants...

It was worse than that. What they actually built was a system designed to support complex hybrid structuring. It's what markets desks had been making a lot of money in prior to the crash esp GS. Unfortunately, post-crash there wasn't much money in structuring so the Front Office was more interested in investing in flow. Quartz was really, really bad at flow.

It took a long time (and the departure of Mike, Kirat et al) to get Quartz to a position where it was a reasonably sane FO system for the world as was rather than as it had been.

Fun times.

Re: An oral history of Bank Python

#117

Post by grvdrm » Thu, Nov 04, 2021, 1:00 PM UTC

klelatti wrote: post #110

grvdrm wrote: post #109

I'm curious about experiences in other similar orgs: I work as a portfolio manager for a large reinsurance/insurance company but spend significant time in SQL, Python, Excel (not unexpected I'm sure). The Wapole platform in the article struck a chord with me. We built something roughly similar - call it Trek - that handles jobs. Jobs encompass lots of tasks - reading/writing from Excel, executing SQL, running Python,…

Really interesting comments. I'd guess that you have proprietary (e.g. valuation / capital calculation) systems that need to interface with Trek in some way. Could you share how you've approached that at all? Also not clear why R couldn't be added to Trek alongside Python and C#?

Certainly - let me try to share succinct version germane to my day-to-day

- We regularly perform group/segment level risk roll-ups. Involves running computationally expensive (by insurance standards) in-house and third-party models that estimate loss from hurricanes, earthquakes, etc. A lot of our insurance data is unsurprisingly stored in disparate systems that don't talk to each other, and in some cases, don't have any useful interface that someone like me can query/view. Things are changing, but still quite a lot of history to overcome

- We also have lots of stuff from outside underwriting parties in form of Excel, CSV, MDF files.

- We have to bring all that together to make sense of the portfolio, so we use Trek to do a lot of the various involved tasks like running the models, processing CSV data, processing Excel data, attaching databases, creating dashboards in PowerBI (tangent: hate it)

- Sample pipeline: query portfolio data from one DB, read CSV file from another third-party, pull both into risk model, kick off analysis, then execute a script to pull together results in the model's databases or elsewhere.

Happy to expand or answer other q's as you have them.

As for your other comment about R - it's just a matter of the install. Someone has to install R so that Trek can use it. Not a major problem. Pointed this out as a contrast in our org that is probably smaller and has many fewer devs compared to what I'm reading in the post where "bank python" sort of feels like the platform in which everything happens / everything is configured.

Re: An oral history of Bank Python

#118

Post by lucozade » Thu, Nov 04, 2021, 1:04 PM UTC

benjaminwootton wrote: post #10

This pattern in large part came from SecDB at Goldman, and then a few people who moved to moved to JPMC and BAML. Dependency graphs are an elegant solution to risk management and pricing etc. There’s a reason this approach works in IBanks. Check out Beacon.io which is the a SaaS implementation from the same team.

> Dependency graphs are an elegant solution to risk management and pricing etc.

Dependency graphs are not a solution to risk and pricing. They are, in certain circumstances, a very useful tool. That's all. They also scale notoriously painfully.

Putting a dependency graph as a mandatory component in your risk system was one of the worst technical decisions I've come across (and I've been doing this lark a long time).

Re: An oral history of Bank Python

#119

Post by vba » Thu, Nov 04, 2021, 1:09 PM UTC

seanhunter wrote: post #21

I was the person who first deployed Python at Goldman Sachs. At the time it was an "unapproved technology" but the partner in charge of my division called me and said (This is literally word for word because partners didn't call me every day so I remember) Err....hi Sean, It's Armen. Uhh.... So I heard you like python.... well if someone was to uhh.... install python on the train... they probably wouldn't be fired. O…

What year was this out of interest?

Re: An oral history of Bank Python

#120

Post by musiciangames » Thu, Nov 04, 2021, 1:19 PM UTC

twic wrote: post #61

nraynaud wrote: post #13

it has a bit of a smalltalk flavor, where the runtime is a memory image, with objects and data in a giant jumble.

There was also a giant investment bank system written in Smalltalk, so that may be a direct influence: http://www.esug.org/data/ESUG2004/ValueOfSmalltalk.pdf

Can anyone confirm whether JP Morgan were able to decommission Kapital when they went to Athena? I've seen so many cases in banks where the old system is still running years and years after it was 'replaced'. And Kapital was used in so many, different, parts of the business.