Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An oral history of Bank Python

calpaterson.com

Post reply on HN
Print view View original

101–110 of 333 posts

Re: An oral history of Bank Python

#101

Post by cbzbc »

curiousgal wrote: post #77
simonh wrote: post #72

Earlier quoted context omitted.

I think Minerva is clearly a reference to Athena.

Could be a misdirection because all of the rest fits Quartz to a tee. The Quartz database is called Sandra (referred to as Barbara here). The Quartz directed acyclic graph is called Dag (referred to as Dagger here) The Quartz job runner is called Bob (referred to as Walpole here which is a reference to Robert Warpole whose shortname is..Bob) These and the horrible proprietary IDE make it obvious which particular syst…

How are the Barbara databases synchronized - as multiple nodes are mentioned ? The description makes it sound like it's just a large set of pickles in something like a Berkley DB?

Re: An oral history of Bank Python

#102

Post by brazzy »

carnitine wrote: post #58

Always funny to see the objections of new hires without finance experience to the use of floating point for pricing. It’s more related to the inherent inaccuracy of any pricing model though, rather than clients not caring about pennies.

Exactly. Floating point is inappropriate for anything related to accounting (payments, balances, etc.), but when the numbers you're producing are effectively forecasts or estimates, it's no different than what floats were originally invented for (numerical modelling in physics and engineering).

Re: An oral history of Bank Python

#103

Post by klelatti »

> This kind of Big Enterprise technology however takes away that basic agency of those Excel users, who no longer understand the business process they run and now has to negotiate with ludicrous technology dweebs for each software change. The previous pliability of the spreadsheets has been completely lost.

> Financiers are able to learn Python, and while they may never be amazing at it they can contribute to a much higher level and even make their own changes and get them deployed.

Coming from a slightly different part of the finance world (insurance) this rang very true.

I think there is a huge opportunity here to build on the Python ecosystem - which is gaining more and more ground - and provide much more powerful alternatives to Excel and legacy proprietary systems.

Re: An oral history of Bank Python

#104

Post by brazzy »

JackFr wrote: post #100
carnitine wrote: post #58

Always funny to see the objections of new hires without finance experience to the use of floating point for pricing. It’s more related to the inherent inaccuracy of any pricing model though, rather than clients not caring about pennies.

I remember someone demanding that we needed to run our Monte Carlo pricer on 1024 paths, that 256 just wasn’t precise enough and one of the risk guys said “Well since we know our assumptions are wrong I’m not sure what difference it makes.”

The difference between precision and accuracy in a nutshell.

Re: An oral history of Bank Python

#105

Post by urban_winter »

odiroot wrote: post #98
urban_winter wrote: post #53

BAML Quartz was conceived by a bunch of front-office quants who had not the first idea about the software needs of a big bank beyond the front office. There was an arrogant assumption that front office software is obviously the most complicated/difficult variety of software within a bank and therefore any system designed with front office requirements at the forefront would, of course, be perfect for universal use. T…

Can you have a career in finance as an engineer with Python and without C/C++ (professional) experience? Your post really made think it, it's an attractive area to work in.

Yes. Many adverts will specify financial services experience but it's worth applying anyway. You'll probably find that roles in back-office technology areas (operations, finance etc) are less demanding in this respect. I hired mostly from outside the financial services industry because other industries had, on average, better-skilled developers, lower salaries and better development practices.

Re: An oral history of Bank Python

#106

Post by kwertyoowiyop »

simonh wrote: post #41
rich_sasha wrote: post #24

Earlier quoted context omitted.

I think it’s mostly true. The complaint with this kind of “industrial global Python code base”, be it at banks or elsewhere, is that often they are hastily cobbled together and depend on extreme user care to not flop over all the time. I guess banks are the archetypal places that care only about feature creation and not about maintenance or technical debt. When something does break in the end, someone senior just sho…

You could not be more wrong. Quartz has first class documentation, solid tooling, very well thought out and rigorous code review and access controls. Banks are regulated up to the eyeballs, so everything has to be audited and justified in detail. It's not nirvana, these are real working systems built by humans with human failings. There are tradeoffs. Not every application is suited to these sorts of platforms, but t…

For some reason people think everything is as good as it can be at FAANG and other big name tech companies, and everyone else just walks around with their pants down at their ankles bumping into walls until 5:00. It’s just not true.

Re: An oral history of Bank Python

#107

Post by markus_zhang »

alecco wrote: post #86

I worked at one of the largest of these systems. It seems to be the one referred by the post. The global distributed store of pickled python objects using Event Sourcing was one of the most horrible and expensive database systems I've ever heard of. It runs on THOUSANDS of expensive servers with all data stored in-memory. To get the state of a single deal you had to open, decompress, deserialize, and merge hundreds i…

But it was good for the first generation who worked there. Big dollars, total control and even the opportunity to create a proprietary IDE...

Re: An oral history of Bank Python

#108

Post by kwertyoowiyop »

captainmuon wrote: post #18

Reminds me of what we used at the ATLAS experiment at CERN* . Python was tightly integrated with the application framework, Athena (which I just realize has the same name as JPM's Python framework!). You could use it as a job description language, and you would compose computation steps from classes you could write in C++. I think there was a separate `athena` executable that was just python with some packages pre-lo…

The Greek and Roman gods have always been a go-to for project names, LOL. We need to give some other cultures a shot!

Re: An oral history of Bank Python

#109

Post by grvdrm »

I'm curious about experiences in other similar orgs:

I work as a portfolio manager for a large reinsurance/insurance company but spend significant time in SQL, Python, Excel (not unexpected I'm sure).

The Wapole platform in the article struck a chord with me. We built something roughly similar - call it Trek - that handles jobs. Jobs encompass lots of tasks - reading/writing from Excel, executing SQL, running Python, running C#. I could list many limitations, but realistically, the biggest limitation is that the platform can't handle something that it can't configure and run. In other words, the platform isn't set up with R - so no creates data pipelines/jobs that use R. Lots of people here use R (among other tools)

One key problem (maybe?) is this all action happens inside the business. Trek was built by a talented actuary/programmer. No software engineering org involvement at all. I'm sure lots of folks here can imagine why: lots of red tape, general adversity to software that isn't already here, long stretches of time to get things done. Also, frankly, lots of our software devs write bad code.

For folks familiar with the orgs in this article, and other similar orgs, is what the article discusses happening mostly with software devs in IT functions? Are these folks embedded in the business? And also are there folks using the more technical bits of the systems that are business-oriented - analysts, investment professionals, etc.

Realize the lines are very blurry these days, but interested to learn from everyone here about the types/roles of end-users

Re: An oral history of Bank Python

#110

Post by klelatti »

grvdrm wrote: post #109

I'm curious about experiences in other similar orgs: I work as a portfolio manager for a large reinsurance/insurance company but spend significant time in SQL, Python, Excel (not unexpected I'm sure). The Wapole platform in the article struck a chord with me. We built something roughly similar - call it Trek - that handles jobs. Jobs encompass lots of tasks - reading/writing from Excel, executing SQL, running Python,…

Really interesting comments.

I'd guess that you have proprietary (e.g. valuation / capital calculation) systems that need to interface with Trek in some way. Could you share how you've approached that at all?

Also not clear why R couldn't be added to Trek alongside Python and C#?

Post reply on HN