Live data from Hacker News

An oral history of Bank Python

calpaterson.com

261–270 of 333 posts

Re: An oral history of Bank Python

#261
post #258

Earlier quoted context omitted.

Armen was very passionate about the value of "strats" (GS's own term for "quants", and later broadened to include software engineers and data scientists). A favorite quip of his: At GS, I'm like an arms dealer. When a desk has a problem, I send in the strats, and they blow away all the competition! Also, SecDB's core idea is not just tight integration between the backend and development environment, but that all obje…

This immediately struck me when I was reading this article. To be honest, this whole paradigm seems absurdly fucking efficient for the developers. But I wonder about stuff like * What happens if the data model needs to change? If you need to move something from db["some/path"]? * How is it coordinated at a larger scale, how does everyone know what is running and how it interacts with everything else - can you figure…

> What happens if the data model needs to change?

You write conversions and there's a registry where you register them to be picked up by the unpickler. If necessary you can also customize the logic that determines which version a given pickled datum uses to deserialize. There aren't so many guardrails when you're writing that stuff, but the infrastructure does its best to support you.

> If you need to move something from db["some/path"]?

There's support for both symlinks (db["some/path"] -> db["other/path"]) and for a kind of hardlink by making both paths point to the same inode-line id. You can usually find a way to do what you need to.

> How is it coordinated at a larger scale, how does everyone know what is running and how it interacts with everything else - can you figure out what depends on an object? What if the data used by your Price(Security) object changes and breaks it?

There's a common model for the things that are shared, and that has a versioning and release/deprecation cycle. Otherwise every type has an owner and you probably had to request their permissions to read their data, so you should have a channel of communication with them. But yeah people do rely on the fundamental business entities not changing too quickly, and things do break when changes are made.

Re: An oral history of Bank Python

#263
post #255
post #247

Earlier quoted context omitted.

Exactly, so what is this Armen character getting out of this other than a potentially big amount of liability and unarticulated risk. The OP said he told people openly that Armen told him he could do it when asked. This makes no sense to me, what’s the upside to Armen? If he is business savvy, he needs to be gaining something in exchange for having his name thrown around by OP as signing off on this.

Guess the Python was useful? From an FT article: >In 2011 Goldman Sachs put its top computer wizard, Armen Avanessians, in charge of the division. He has helped turn round its fortunes. The arm’s assets under management reached a nadir of $38bn in 2012, but it now manages $91.8bn...

Wow hope OP got a chunk of that!

Re: An oral history of Bank Python

#264
post #197

Earlier quoted context omitted.

Curious why would this high-powered person go to bat for a technology decision they didn’t seem to have done any risk assessment of? Wouldn’t he be liable if something was exploited and hurt and company, like his head would be on the chopping block for giving the go ahead when they traced it up the chain of command?

That's precisely why he did it the way he did. He had total deniability. Here's how the conversation would go if it went wrong somehow: "Armen, did you tell Sean to install python?" "No". "Sean, did Armen tell you to install python?" "Err... no. He said I probably wouldn't be fired." "Well it turns out he's not right about everything. Here's a cardboard box for your things."

Nah, I don't think that's why he did it.

You say he reached out to you and asked if you liked Python. He probably wanted to roll out Python and was looking for someone who wanted to do it. If he told someone to do something they weren't passionate about, they would fail. He wanted to make sure it succeeded, so he reached out to you.

If he's such a bigshot and everyone was frightened of him, he must not have been afraid of them. When he said, "you wouldn't get fired," he probably meant what he said. He was giving you air cover. And it worked. When the gnomes came out after you, you just sent them to him. And they didn't bother you again.

I can imagine how the conversation went:

"Armen, did you tell Sean to install Python"

"No, did he"?

"Yes he did!"

"Great!!"

Now the gnomes are on their backfoot and have to defend why Sean shouldn't install Python If this guy Armen told you to do it, Armen has to defend himself to them.

Re: An oral history of Bank Python

#265

There's a lot here in common with the higher-prestige systems operated by major tech companies. Giant custom monorepos, sometimes with custom IDEs built into them. Big proprietary services for running asynchronous jobs and collecting logs and everything else. Data-driven frameworks for spinning up new services. Bespoke databases. All of it rings a bell. The one thing in there that really jumped out at me as "oh my go…

It’s not just straight pickle (usually) - there’s a layer in between that allows at the very least for the handling of deprecated fields/new fields/renames etc. But - after a change - you have to choose either to leave that ‘backward compatibility’ in-place (essentially forever), or put together a job to run (on that scheduler! Hah) to go re-write in your new format. If you care enough, you might - and then you can r…

> there’s a layer in between that allows at the very least for the handling of deprecated fields/new fields/renames etc.

That sounds pretty reasonable. Not so different from tossing JSON objects into MongoDB or something like that.

Re: An oral history of Bank Python

#266
post #263
post #255

Earlier quoted context omitted.

Guess the Python was useful? From an FT article: >In 2011 Goldman Sachs put its top computer wizard, Armen Avanessians, in charge of the division. He has helped turn round its fortunes. The arm’s assets under management reached a nadir of $38bn in 2012, but it now manages $91.8bn...

Wow hope OP got a chunk of that!

It's a bank, so probably his reward for good performance was 10k at Christmas.

Re: An oral history of Bank Python

#267
post #31

Earlier quoted context omitted.

I worked on Quartz for 3 years and loved it. Some devs grumbled about various aspects of it, but I come from an application support background and taught myself python, so I suppose I had fewer developer habits to un-learn. From what I understand, all this started with SecDB at Goldman, which was a the prototype for all these systems but wasn't Python based. The lore is that SecDB was instrumental in Goldman being ab…

I work at Beacon.io and it's an awesome place to be. Kirat is indeed a rockstar and it's awesome to work with an CEO that knows great code. We also landed a Series C last month and we're growing :) https://www.crunchbase.com/funding_round/beacon-series-c--60... We've also got a bunch of positions open too for those that are interested in joining! https://www.beacon.io/careers/

As a full stack quant dev who is struggling to go all-in on Beacon: Do people generally like to use Glint for complex applications? Are benefits of Beacon lost when interfacing from e.g. Angular? I'm afraid that professional frontend devs might be unwilling to work with a proprietary framework, but that's speculation from my side.

Re: An oral history of Bank Python

#268

Really interesting read. From what's described: Walpole (distributed job runner), Dagger (DAG that recalculates when dependencies change), Barbara (global key value store) and monorepo/fast deployment, its not so different from some big tech companies.

You can see how it's tied together in this presentation https://youtu.be/M9o9SF5-Pzw

Re: An oral history of Bank Python

#269
post #256

Earlier quoted context omitted.

Does everyone use a giant Pickle dump ? I mean - how big is that ? Petabytes ? I'm kind of surprised nobody monkey patched python serialisation to use a database (much like GitHub did with ssh key lookup in MySQL). What does the devops there look like ? Snapshot every minute ?

It's not a single giant pickle dump; each individual object gets pickled and stored in Minerva (which works more or less like Cassandra or something). It's a pretty similar high level design to what the likes of Google or Facebook do do where you store everything as protobufs in BigTable - the bank uses pickle rather than protobuf because they put a higher priority on being able to store arbitrary objects and deal wi…

So Minerva is like a distributed datastore, specifically for python object storage ?

Interesting. Do you think you would do this today with a Cassandra/Hbase? Can it be done - let's say take python 3.10 and the latest Cassandra (or even better - something like Firebase or Cloud Spanner).

Just curious that in a post AWS/Firebase world, can something like Minerva be built, without investing in writing the db store ground up.

Post reply on HN