Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An oral history of Bank Python

calpaterson.com

Post reply on HN
Print view View original

211–220 of 333 posts

Re: An oral history of Bank Python

#211

Post by chewbaxxa »

Chris2048 wrote: post #193
carnitine wrote: post #171

Earlier quoted context omitted.

You don’t do that though, there’s no use case to account with the output of a model.

I'm not sure what you mean - aside from speculative pricing models there are regulatory constraints too, that are a part of the same codebase. Not to mention that there is a use case for auditing pricing models (as in external requirement, or internally) or comparing alternative models.

Sure we add up the numbers, but there are thresholds for everything anyway. We might not be concerned about something within a 1MM range let alone a floating point inaccuracy. The uncertainty is accounted for already.

Re: An oral history of Bank Python

#212

Post by seanhunter »

simonh wrote: post #31
seanhunter wrote: post #21

I was the person who first deployed Python at Goldman Sachs. At the time it was an "unapproved technology" but the partner in charge of my division called me and said (This is literally word for word because partners didn't call me every day so I remember) Err....hi Sean, It's Armen. Uhh.... So I heard you like python.... well if someone was to uhh.... install python on the train... they probably wouldn't be fired. O…

I worked on Quartz for 3 years and loved it. Some devs grumbled about various aspects of it, but I come from an application support background and taught myself python, so I suppose I had fewer developer habits to un-learn. From what I understand, all this started with SecDB at Goldman, which was a the prototype for all these systems but wasn't Python based. The lore is that SecDB was instrumental in Goldman being ab…

> From what I understand, all this started with SecDB at Goldman, which was a the prototype for all these systems but wasn't Python based. The lore is that SecDB was instrumental in Goldman being able to rapidly figure out what their exposure was during the 2008 crisis.

Correct. We used python for a bunch of infrastructure stuff (eg distributing all of secdb to all of the places it needed to go). The actual pricing and risk was written in Slang with a lot of guis that were "written" in slang but actually that caused autogeneration of JIT bytecode that was executed by a JVM. Most of the heavy lifting behind the scenes was C++. So a bit of everything.

Re: An oral history of Bank Python

#213

Post by databag »

forgotmyoldacc wrote: post #22

Really interesting read. From what's described: Walpole (distributed job runner), Dagger (DAG that recalculates when dependencies change), Barbara (global key value store) and monorepo/fast deployment, its not so different from some big tech companies.

At JPM, Athena: Walpole = Bob job runner Dagger = pixies Barbara = hydra

The python 3 migration is still ongoing

Re: An oral history of Bank Python

#214

Post by harel »

Chris2048 wrote: post #129
harel wrote: post #44

I worked on Quartz for a while as a contractor. Hated every second of it. Python version was old (2.4 if I remember correctly when 3.x was already the popular version). But that wasn't it. It was the proprietary version of everything in the stack that got me. Proprietary ide, source control, libs etc. I noted that none of the others who have been there for years have any transferable skills that can cary them out of…

> Proprietary ide, source control, libs etc The reason this was a problem was that it meant investment was needed for each of these things, and as such fell behind. The IDE fell behind most modern IDEs, presumably because it didn't get the budget for it. The source control/ libs where usually modified versions of existing libs, but now needing to be maintained internally to remain compatible with the mainstream versi…

Yes, it is their fault, but the organisation didn't even attempt to nurture professional development. Stagnation was a feature, not a bug. Arguably, these guys were paid very well so they would have taken a pay cut anywhere else anyway.

Re: An oral history of Bank Python

#215

Post by harel »

Chris2048 wrote: post #129
harel wrote: post #44

I worked on Quartz for a while as a contractor. Hated every second of it. Python version was old (2.4 if I remember correctly when 3.x was already the popular version). But that wasn't it. It was the proprietary version of everything in the stack that got me. Proprietary ide, source control, libs etc. I noted that none of the others who have been there for years have any transferable skills that can cary them out of…

> Proprietary ide, source control, libs etc The reason this was a problem was that it meant investment was needed for each of these things, and as such fell behind. The IDE fell behind most modern IDEs, presumably because it didn't get the budget for it. The source control/ libs where usually modified versions of existing libs, but now needing to be maintained internally to remain compatible with the mainstream versi…

It's not just the "knowing some lib". It's a way of working that is not compatible with the outside world. I had every single character i type having to get director approval. I sopped counting broken things that required human intervention (on rota). The style of programming is... well.. . bankish? I would have had to take a big gamble hiring most of these guys.

Re: An oral history of Bank Python

#216

Post by noneeeed »

seanhunter wrote: post #76
simonh wrote: post #31

Earlier quoted context omitted.

I worked on Quartz for 3 years and loved it. Some devs grumbled about various aspects of it, but I come from an application support background and taught myself python, so I suppose I had fewer developer habits to un-learn. From what I understand, all this started with SecDB at Goldman, which was a the prototype for all these systems but wasn't Python based. The lore is that SecDB was instrumental in Goldman being ab…

I know Kirat really well. Fun fact, one two-week dev cycle we had 667 distinct developers commit to the secdb code base which Kirat's boss described to me as "The number of the beast.... plus Kirat" Second fun thing. Kirat was advocating for lisp for secdb for a long time and used to rag on me for liking python when it's so slow.

Interesting to be reading all this about SecDB. About 15 years ago I was offered a job working on SecDB (I forget exactly what the position was now). It and Slang sounded really interesting.

I do sometimes regret not taking the job because the people there were wickedly sharp and the tech sounded great, but in hindsight I'm not sure I would have thrived in a bank long term. I did a 3 month internship at Lehman's which I enjoyed, but I don't think I'd have suited a career in it. One thing I did get out of it was a total lack of fear around job interviews, if I could survive the 14 hours of interviews at GS and come out with an offer, then I can handle pretty much any recuitment process :)

Re: An oral history of Bank Python

#217

Post by redsid »

I worked on modeling/mapping market risk schema in Quartz a few years ago and used to wonder why they were "customizing" open source software/systems in house, when they can as well as supported those initiatives directly and publicly. As a C++ dev,I had already realized the world of software tooling had passed by me, but still used to wonder at the (over)engineering of everything in quartz and involved skills that were not transferable.

My view now is the value of these in-house systems is essentially a cost efficiency play on run costs, and there there is very little revenue/growth opportunities for the business from these investments. With Volcker (really the best thing to happen in the '10s) and loss of prop trading means all the market makers live off the spread, and so while there is value to minimize operational costs, they are not worth the investments that have been made.

Bottom line for me is investments within large firms in capital markets are unlikely to generate revenue/profits in scale - I am sure there are some exceptions and would like to know

Re: An oral history of Bank Python

#218

Post by markus_zhang »

mtrovo wrote: post #152

> I've mentioned that programmers are far too dismissive of MS Excel. You can achieve a awful lot with Excel: more, even, than some programmers can achieve without it This is one of the most underrated topics in tech imho. Spreadsheet is probably the pinnacle of how tech could be easily approachable by non tech people, in the "bike for the mind" sense. We came a long way down hill from there when you need an speciali…

The problem is sometimes analysts turn into shadow BI or even DE who only know Excel. They know Excel so well that they create a whole monstrosity in Excel. MSFT has been sort of encouraging that too by introducing some Power BI feature and now Javascript into Excel.

Re: An oral history of Bank Python

#219

Post by smallnamespace »

seanhunter wrote: post #21

I was the person who first deployed Python at Goldman Sachs. At the time it was an "unapproved technology" but the partner in charge of my division called me and said (This is literally word for word because partners didn't call me every day so I remember) Err....hi Sean, It's Armen. Uhh.... So I heard you like python.... well if someone was to uhh.... install python on the train... they probably wouldn't be fired. O…

Armen was very passionate about the value of "strats" (GS's own term for "quants", and later broadened to include software engineers and data scientists).

A favorite quip of his:

  At GS, I'm like an arms dealer. When a desk has a problem, I send in the strats, and they blow away all the competition!
Also, SecDB's core idea is not just tight integration between the backend and development environment, but that all objects ("Security" in SecDB lingo) were functionally reactive.

For example, you would write a pure function that defines:

  Price(Security) = F(stock price, interest rate, ...)
When the market inputs changed, Price(Security) would automatically update (the framework handled the fiddly bits of caching intermediate values for you, so even an expensive Price function is not problematic).

This is loosely the same idea that drives React, ObservableHQ, Kafka, and other event-streaming architectures, but I first encountered this ~15 years ago at a bank.

Re: An oral history of Bank Python

#220

Post by markus_zhang »

jaromir_ wrote: post #210
harel wrote: post #44

I worked on Quartz for a while as a contractor. Hated every second of it. Python version was old (2.4 if I remember correctly when 3.x was already the popular version). But that wasn't it. It was the proprietary version of everything in the stack that got me. Proprietary ide, source control, libs etc. I noted that none of the others who have been there for years have any transferable skills that can cary them out of…

I second your opinion (interned @ Athena, not Quartz) - compared to my current BigN experience everything was worse by a magnitude: the IDE, the source control, the review mechanism, the job scheduler and so on. I'd expect with so many devs working on this the DevX will be ironed out

I wonder how did they make the IDE? Must be an interesting job for whoever got to write it, and hell for whoever is maintaining it and using it, lol.
Post reply on HN