221–230 of 333 posts
Really interesting read. From what's described: Walpole (distributed job runner), Dagger (DAG that recalculates when dependencies change), Barbara (global key value store) and monorepo/fast deployment, its not so different from some big tech companies.
At JPM, Athena: Walpole = Bob job runner Dagger = pixies Barbara = hydra The python 3 migration is still ongoing
> I've mentioned that programmers are far too dismissive of MS Excel. You can achieve a awful lot with Excel: more, even, than some programmers can achieve without it This is one of the most underrated topics in tech imho. Spreadsheet is probably the pinnacle of how tech could be easily approachable by non tech people, in the "bike for the mind" sense. We came a long way down hill from there when you need an speciali…
problem as described to me is that excel starts being used for regulated processes and it's not well auditable, access controlled, changed controlled, tracked, etc etc. Then people need to implement the exact same process across departments and they're all using a separate excel sheet and they all submit different numbers. becomes a huge mess and so much more complicated and expensive systems become commissioned.
Earlier quoted context omitted.
Can you have a career in finance as an engineer with Python and without C/C++ (professional) experience? Your post really made think it, it's an attractive area to work in.
Absolutely yes. Depending on what kind of engineer, it is far better to go to the finance (front office quant, back office risk) side than the tech support side. They are less snobbish about autodidacts and pay is far better if you are willing to learn about things outside the dev sandbox. (Our front office has a few quants and ex-quants with electric engineering background, I don't know of any software engineers the…
I can see the benefits of this collection of tools within an all-in-one monolith. Ease of deployment is a big benefit. I can also see the costs. As a stack its probably better in some ways than how a lot of other businesses operate as well as worse. There's probably a lot both ways. The mainframe mindset might be a factor here as well. The giant mainframe where all the magic happens is still a thing to behold and thi…
> The part about programming languages frequently not having tables is interesting. The closest as mentioned is the hash, but you lose so much in that abstraction eg the relational aspects. The counter argument then becomes the obvious: why aren't you using a database library, or in a pinch, sqlite? Rightly so. Why would you add relational tables to python rather than have a generic python database spec or a collecti…
And yes, it made for some very powerful high-level tooling. I actually learned to code on FoxPro for DOS, and the efficiency with which you could crank out even fairly complicated line-of-business data-centric apps was amazing, and is not something I've seen in any tech stack since.
Earlier quoted context omitted.
Curious why would this high-powered person go to bat for a technology decision they didn’t seem to have done any risk assessment of? Wouldn’t he be liable if something was exploited and hurt and company, like his head would be on the chopping block for giving the go ahead when they traced it up the chain of command?
That's precisely why he did it the way he did. He had total deniability. Here's how the conversation would go if it went wrong somehow: "Armen, did you tell Sean to install python?" "No". "Sean, did Armen tell you to install python?" "Err... no. He said I probably wouldn't be fired." "Well it turns out he's not right about everything. Here's a cardboard box for your things."
Armen must have known people would know Python had been put on these machines and that he authorised it, in fact what's the point of putting it on them if nobody knows and nobody uses it? I can guarantee you that within 24 hours someone was asking Armen why he'd authorised this and was justifying it. There cannot have been any possibility of dodging responsibility for this decision. If anyone got fired it would have been Armen, with a possibility of Sean going as collateral damage.
This is the big league. You make your decisions and you accept responsibility for them.
Earlier quoted context omitted.
Or they could use instead use CSVs. What could possibly go wrong?
Everything is fine, as long as no Americans come and write 1,000 where obviously they should have written 1000 or 1'000. /s
Earlier quoted context omitted.
Hmm, I found the opposite - the fact that there was this global framework that managed all the data and code meant that access control was actually pretty good, better than most tech companies I've worked for. You had a single source of truth for what your access rights were, there was integrated Kerberos any time you needed to access a system outside Minerva. And having all the code in a managed place meant good dep…
This was 100% my experience too. The biggest productivity gains were: - having a single source of truth for both data and code (in a closely coupled environment) - strong, battle-tested libraries to take care of all infrastructure concerns. - enforced code dev/test/review/deployment workflows This let the front-office devs be highly productive on adding real business value for their trading desks. Remember also that…
GS had „these systems“ well before 2000 (via J Aron). I think around the time you mentioned they spread to other firms (in their Python reincarnation).
The one thing in there that really jumped out at me as "oh my god never ever do that" was using pickle for serious persisted data. I can see the upside around letting people who aren't dedicated programmers avoid thinking about serialization but...daaamn. My understanding is that this locks you into a "the API is all the code" situation where you can never change the data layout of any class once you've written it?
I was the person who first deployed Python at Goldman Sachs. At the time it was an "unapproved technology" but the partner in charge of my division called me and said (This is literally word for word because partners didn't call me every day so I remember) Err....hi Sean, It's Armen. Uhh.... So I heard you like python.... well if someone was to uhh.... install python on the train... they probably wouldn't be fired. O…
Armen was very passionate about the value of "strats" (GS's own term for "quants", and later broadened to include software engineers and data scientists). A favorite quip of his: At GS, I'm like an arms dealer. When a desk has a problem, I send in the strats, and they blow away all the competition! Also, SecDB's core idea is not just tight integration between the backend and development environment, but that all obje…
I built a similarly reactive system for web UI binding back in 2004, running binding expressions on the back end with cached UI state to compute the minimal update to return to the front end, in the form of attributes to set on named elements.