Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An oral history of Bank Python

calpaterson.com

Post reply on HN
Print view View original

321–330 of 333 posts

Re: An oral history of Bank Python

#321

Post by KMag »

lmm wrote: post #261
jb_s wrote: post #258

Earlier quoted context omitted.

This immediately struck me when I was reading this article. To be honest, this whole paradigm seems absurdly fucking efficient for the developers. But I wonder about stuff like * What happens if the data model needs to change? If you need to move something from db["some/path"]? * How is it coordinated at a larger scale, how does everyone know what is running and how it interacts with everything else - can you figure…

> What happens if the data model needs to change? You write conversions and there's a registry where you register them to be picked up by the unpickler. If necessary you can also customize the logic that determines which version a given pickled datum uses to deserialize. There aren't so many guardrails when you're writing that stuff, but the infrastructure does its best to support you. > If you need to move something…

There's also a graph debugger that allows you to step through the dependency graph node-by-node across the various globally distributed databases.

Re: An oral history of Bank Python

#322

Post by oogetyboogety »

stevesimmons wrote: post #57
seanhunter wrote: post #21

I was the person who first deployed Python at Goldman Sachs. At the time it was an "unapproved technology" but the partner in charge of my division called me and said (This is literally word for word because partners didn't call me every day so I remember) Err....hi Sean, It's Armen. Uhh.... So I heard you like python.... well if someone was to uhh.... install python on the train... they probably wouldn't be fired. O…

I worked on Athena at JPMorgan for 8 years, and loved it. Seeing Python at the core of trading, risk and post-trade processing for Commodities, FX, Credit etc was such a great developer experience. By the time I left JPM, there were 4500 devs globally making 20k commits weekly into the Athena codebase. (I did a PyData presentation on this [1] for more details). The one downside was the delayed transition from Py2.7 t…

That's funny they mentioned replayable financial message queues. Those are a hit here

Re: An oral history of Bank Python

#323

Post by lmm »

KMag wrote: post #321
lmm wrote: post #261

Earlier quoted context omitted.

> What happens if the data model needs to change? You write conversions and there's a registry where you register them to be picked up by the unpickler. If necessary you can also customize the logic that determines which version a given pickled datum uses to deserialize. There aren't so many guardrails when you're writing that stuff, but the infrastructure does its best to support you. > If you need to move something…

There's also a graph debugger that allows you to step through the dependency graph node-by-node across the various globally distributed databases.

True but not really helpful for this problem, because it can only tell you about the job you're debugging, whereas what you want to know is what code might ever depend on that data.

Re: An oral history of Bank Python

#324

Post by chestervonwinch »

smallnamespace wrote: post #219
seanhunter wrote: post #21

I was the person who first deployed Python at Goldman Sachs. At the time it was an "unapproved technology" but the partner in charge of my division called me and said (This is literally word for word because partners didn't call me every day so I remember) Err....hi Sean, It's Armen. Uhh.... So I heard you like python.... well if someone was to uhh.... install python on the train... they probably wouldn't be fired. O…

Armen was very passionate about the value of "strats" (GS's own term for "quants", and later broadened to include software engineers and data scientists). A favorite quip of his: At GS, I'm like an arms dealer. When a desk has a problem, I send in the strats, and they blow away all the competition! Also, SecDB's core idea is not just tight integration between the backend and development environment, but that all obje…

> This is loosely the same idea that drives React, ObservableHQ, Kafka, and other event-streaming architectures, but I first encountered this ~15 years ago at a bank.

See also the "observer pattern" [0]. It's a fun exercise to implement a reactive system in Python using the descriptor protocol [1]. IPython's traitlets library is an example of this in the wild [2].

[0]: https://en.wikipedia.org/wiki/Observer_pattern

[1]: https://docs.python.org/3/howto/descriptor.html

[2]: https://github.com/ipython/traitlets

Re: An oral history of Bank Python

#325

Post by webmaven »

Huh. A lot of Bank Python sounds eerily similar to Zope related tech:

'Barbara' (which I suspect is JP Morgan's Hydra) sounds like a brutalist version of the ZODB created with scalability in mind.

The hierarchical overlays sound like Zope's Acquisition and Containment.

Even storing (and running) code in the object database has it's equivalent in Zope's Python Script Objects.

Re: An oral history of Bank Python

#326

Post by swuecho »

chestervonwinch wrote: post #324
smallnamespace wrote: post #219

Earlier quoted context omitted.

Armen was very passionate about the value of "strats" (GS's own term for "quants", and later broadened to include software engineers and data scientists). A favorite quip of his: At GS, I'm like an arms dealer. When a desk has a problem, I send in the strats, and they blow away all the competition! Also, SecDB's core idea is not just tight integration between the backend and development environment, but that all obje…

> This is loosely the same idea that drives React, ObservableHQ, Kafka, and other event-streaming architectures, but I first encountered this ~15 years ago at a bank. See also the "observer pattern" [0]. It's a fun exercise to implement a reactive system in Python using the descriptor protocol [1]. IPython's traitlets library is an example of this in the wild [2]. [0]: https://en.wikipedia.org/wiki/Observer_pattern […

Thanks for the links. eye opener for me, despite using python for serveral years.

Re: An oral history of Bank Python

#327

Post by ravibala1 »

smallnamespace wrote: post #219
seanhunter wrote: post #21

I was the person who first deployed Python at Goldman Sachs. At the time it was an "unapproved technology" but the partner in charge of my division called me and said (This is literally word for word because partners didn't call me every day so I remember) Err....hi Sean, It's Armen. Uhh.... So I heard you like python.... well if someone was to uhh.... install python on the train... they probably wouldn't be fired. O…

Armen was very passionate about the value of "strats" (GS's own term for "quants", and later broadened to include software engineers and data scientists). A favorite quip of his: At GS, I'm like an arms dealer. When a desk has a problem, I send in the strats, and they blow away all the competition! Also, SecDB's core idea is not just tight integration between the backend and development environment, but that all obje…

Thank you so much for this context! When I started at Lehman in 2007 and for years through the i-banks & hedge-funds, 'Sec-db' was the North Star for so much me and my friends built. It's amazing to hear from the folk who brought that to life!

Took many more years for us to understand that we had learned more about banking and making money than our masters had learned about the potential uses of the platforms we had built.

We started Sandbox Banking. Many of our friends are at hedge-funds :-(. What's the career paths of those that first built sec-db?

Re: An oral history of Bank Python

#328

Post by ravibala1 »

simonh wrote: post #93
JoshTriplett wrote: post #80

Earlier quoted context omitted.

There are a number of workplaces where I'd have been willing to rely on "probably wouldn't be fired", but a bank is definitely not one of them. Congratulations on shipping something useful in the face of that risk and uncertainty.

The trading community walk on a knife edge all the time, it's not a place for the faint of heart. I used to support derivatives trading systems and a few times there were issues that meant they'd lost control of orders on an exchange. Scary stuff. It requires a crazy mixture of careful, deliberate, calculated risk control on the one hand; but once you commit to something you jump in with both feet and throw everythin…

I worked at UBS on the G10 FX options desks in Stamford and Singapore. Remember being very surprised by how my interview and training were incredibly stressful. It was very intentional as well, where my trainer knew the exact reactions he was creating with his behavior.

It was only a couple of weeks in, when I had to react within 60 seconds (USDJPY option expiry, NYC cut) on a position our front book would have lost MM on; with senior sales MDs screaming at me as well. Lo and behold, I was just used to it and could focus and execute based on my training.

My wife calls my thinking on the training - Stockholm syndrome. I still believe those skills were incredibly valuable for me, just perhaps delivered in a more 2021 acceptable approach.

Re: An oral history of Bank Python

#329

Post by secondaryacct »

dolmen wrote: post #312
misja111 wrote: post #59

Earlier quoted context omitted.

> I guess banks are the archetypal places that care only about feature creation and not about maintenance or technical debt. It depends which department you are in, but in general: absolutely not. Actually the reverse is true. Banks have huge risks to manage: just imagine for instance what damage a hack of their account system could cause. Or a crash of their payment system. Therefore it is of the utmost importance t…

And that's why many old banks still use COBOL.

Well it s repeated a lot but I work in a big bank and I have yet to see tech older than Java 6 :(

Re: An oral history of Bank Python

#330

Post by AnonymousPlanet »

bloblaw wrote: post #236
andrewshadura wrote: post #81

Earlier quoted context omitted.

> python build, dependency and deployment management is exceptionally awful in every respect, this isn't as big a pain point in other languages I'm not sure how to react to that, but these features in Python are miles ahead of what many other languages have (or actually don't have).

If you compare Python's deployment and dependency management to those of statically compiled languages like Go, Rust, Zig, or Nim, you quickly see the experience with Python is quite poor. In all the above languages, you simply ship a statically compiled binary (often just 1 file), and the user needs nothing else. With any sufficiently complex Python project, the user will need: 1. virtualenv 2. possibly a C compiler…

Or you just provide your own Python package. Most of the time that will be less than 100 MB if you don't include huge libraries. You can test and build automatically. For deployment you then have an installer or rpm that is probably smaller than most of the other enterprise software your customer's infrastructure admins are handling.
Post reply on HN