Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An oral history of Bank Python

calpaterson.com

Post reply on HN
Print view View original

81–90 of 333 posts

Re: An oral history of Bank Python

#81

Post by andrewshadura »

lmm wrote: post #19

I'm seeing a lot of people speculating about which bank this might be; I think the point is that it's all of them. I could loosely describe a previous job as implementing Morgan Stanley's Walpole and integrating more source code management into Minerva (even though that system wasn't actually Python-based). Having a global view on everything is large banks' value-add, it's why they haven't been outcompeted by their m…

> python build, dependency and deployment management is exceptionally awful in every respect, this isn't as big a pain point in other languages

I'm not sure how to react to that, but these features in Python are miles ahead of what many other languages have (or actually don't have).

Re: An oral history of Bank Python

#82

Post by leprechaun1066 »

Thorrez wrote: post #4

Doesn't seem that strange compared to K[1] or Q[2], which are used by Wall Street banks. K encourages you to use single-letter variables and bunch your code up as tight as possible into long lines. Here's an example: [3]. Interestingly, their Github repo has some K-inspired C[4], Java[5], C#[6], and Javascript[7]. [1] https://en.wikipedia.org/wiki/K_(programming_language) [2] https://en.wikipedia.org/wiki/Q_(programm…

The big banks don't write code in K4 though, managers generally encourage people not to write code in it due to the difficulty of finding developers who are fluent in it.

They all use q and q is very wordy and highly readable if you speak english. It's mostly just developer defined functions which are compositions of the keywords of which there are not many: https://code.kx.com/q/ref/#keywords

Most of the code you would see in a kdb+ system in an investment bank won't look like any of the links you've provided.

Re: An oral history of Bank Python

#83

Post by toyg »

dash2 wrote: post #32

> In order to deploy your app outside of Minerva you now need to know something about k8s, or Cloud Formation, or Terraform. This is a skillset so distinct from that of a normal programmer (let alone a financial modeller) that there is no overlap. This rang a bell. How did deployment become such an arcane skill?

1. Sysadmins had to find new careers after cloud providers destroyed their livelihood.

2. cloud providers try very hard to lock you in, by offering all sorts of advanced goodies. They tend to come with a learning curve, and they all accumulate. Sooner or later someone comes up with cross-provider solutions, and they too have learning curves.

3. inventing new ecosystems means creating new work for advocates and ninjas. You don't become a rockstar by diligently doing what has been done before, but by finding (or inventing) a niche and becoming a guru.

4. some problems are indeed hard to solve, and the more products try to do that, they more they get complex.

5. everyone thinks they will have Facebook-scale problems, even when they never will.

Re: An oral history of Bank Python

#84

Post by Icathian »

Paladiamors wrote: post #74
simonh wrote: post #31

Earlier quoted context omitted.

I worked on Quartz for 3 years and loved it. Some devs grumbled about various aspects of it, but I come from an application support background and taught myself python, so I suppose I had fewer developer habits to un-learn. From what I understand, all this started with SecDB at Goldman, which was a the prototype for all these systems but wasn't Python based. The lore is that SecDB was instrumental in Goldman being ab…

I work at Beacon.io and it's an awesome place to be. Kirat is indeed a rockstar and it's awesome to work with an CEO that knows great code. We also landed a Series C last month and we're growing :) https://www.crunchbase.com/funding_round/beacon-series-c--60... We've also got a bunch of positions open too for those that are interested in joining! https://www.beacon.io/careers/

I took a quick look, it seems like all the postings are London or New York. What's the feeling internally about remote hires? I'm assuming that's still out of fashion in finance and Beacon feels the same?

Re: An oral history of Bank Python

#85

Post by odiroot »

seanhunter wrote: post #21

I was the person who first deployed Python at Goldman Sachs. At the time it was an "unapproved technology" but the partner in charge of my division called me and said (This is literally word for word because partners didn't call me every day so I remember) Err....hi Sean, It's Armen. Uhh.... So I heard you like python.... well if someone was to uhh.... install python on the train... they probably wouldn't be fired. O…

Very interesting story. I'm actually impressed, you're allowed to give us such "internals" about GS.

Re: An oral history of Bank Python

#86

Post by alecco »

I worked at one of the largest of these systems. It seems to be the one referred by the post.

The global distributed store of pickled python objects using Event Sourcing was one of the most horrible and expensive database systems I've ever heard of. It runs on THOUSANDS of expensive servers with all data stored in-memory. To get the state of a single deal you had to open, decompress, deserialize, and merge hundreds if not thousands of instances. And 90% of the output was more often than not discarded.

The Python interpreter extensions reveal the ignorance of Python by the original developers. There was no good reason to fork CPython.

There were many small subsystems created and supported by lone rangers with impressive CVs and astronomical salaries. A JIT better than any other one out there (but with a lot of limitations). A meta-query system extremely elegant.

But this all was a sham. The actual daily crunch/analytics was run on more classic SQL/Columnar clusters. From the distributed object database hours-long running batch jobs loaded stuff on old school DBs. And those blew up frequently. Sometimes those blow ups cost many millions in delayed regulatory reports. The queries running on top of SQL were beyond stupid and the DB engine could not optimize for them. And of course, people blamed SQL and not the ridiculous architecture and the OOP dogma.

Don't work for old school banking, hedge funds, or anything like that. They are driven by tech cavemen and their primadonnas. Exceptions might be some HFT and fintech shops.

Re: An oral history of Bank Python

#87

Post by streamofdigits »

globular-toast wrote: post #54
cstross wrote: post #11

This immediately sprang out at me: > Investment banks have a one-way approach to open source software: (some of) it can come in, but none of it can go out. I wonder how well this plays with the various open source software licenses?

As others have mentioned, it's fine, even with GPL, as the licences only really kick in when they try to distribute the software. They are only really hurting themselves. When starting a private fork you force yourself to maintain it alone. That means either letting it rot (ie. it becomes insecure and obsolete with no new libraries supporting it) or keeping up with the mainstream yourself. Either way it's a lot of wo…

what would be the implication wrt external banking services that use open source software? (echoes the cloud databases story...)

Re: An oral history of Bank Python

#88

Post by op00to »

JoshTriplett wrote: post #80
seanhunter wrote: post #21

I was the person who first deployed Python at Goldman Sachs. At the time it was an "unapproved technology" but the partner in charge of my division called me and said (This is literally word for word because partners didn't call me every day so I remember) Err....hi Sean, It's Armen. Uhh.... So I heard you like python.... well if someone was to uhh.... install python on the train... they probably wouldn't be fired. O…

There are a number of workplaces where I'd have been willing to rely on "probably wouldn't be fired", but a bank is definitely not one of them. Congratulations on shipping something useful in the face of that risk and uncertainty.

Investment banks at that time were a little bit different.

Re: An oral history of Bank Python

#89

Post by Paladiamors »

Icathian wrote: post #84
Paladiamors wrote: post #74

Earlier quoted context omitted.

I work at Beacon.io and it's an awesome place to be. Kirat is indeed a rockstar and it's awesome to work with an CEO that knows great code. We also landed a Series C last month and we're growing :) https://www.crunchbase.com/funding_round/beacon-series-c--60... We've also got a bunch of positions open too for those that are interested in joining! https://www.beacon.io/careers/

I took a quick look, it seems like all the postings are London or New York. What's the feeling internally about remote hires? I'm assuming that's still out of fashion in finance and Beacon feels the same?

Because of Covid I've been remote since last year. It's still an evolving situation. But things have worked out pretty well.

Our CEO was also remote during that time too and here's him giving a webinar from a cabin :)

https://www.youtube.com/watch?v=fXPDXbrPdxI&ab_channel=Beaco....

Re: An oral history of Bank Python

#90

Post by randomcarbloke »

streamofdigits wrote: post #69

> Investment banks have a one-way approach to open source software: (some of) it can come in, but none of it can go out I'd say thats how they see society and their role in it more generally. Doing God's work is a surprisingly directed graph. It applies also to the broader banking world, but investment banks being the most lucrative (when not bailed out) attract talented people that in principle can close the loop an…

it's also not entirely true, I know of some hedge funds that have made significant contributions to open source codebases (including Python)
Post reply on HN