This immediately sprang out at me:
> Investment banks have a one-way approach to open source software: (some of) it can come in, but none of it can go out.
I wonder how well this plays with the various open source software licenses?
An oral history of Bank Python
11–20 of 333 posts
Re: An oral history of Bank Python
#12This immediately sprang out at me: > Investment banks have a one-way approach to open source software: (some of) it can come in, but none of it can go out. I wonder how well this plays with the various open source software licenses?
Quite well. You are under no obligation to commit back your changes under any OSI license. The old Sun CDDL required it, and was denied OSI "open sourceness" as a result.
Re: An oral history of Bank Python
#13it has a bit of a smalltalk flavor, where the runtime is a memory image, with objects and data in a giant jumble.
Re: An oral history of Bank Python
#14Doesn't seem that strange compared to K[1] or Q[2], which are used by Wall Street banks. K encourages you to use single-letter variables and bunch your code up as tight as possible into long lines. Here's an example: [3]. Interestingly, their Github repo has some K-inspired C[4], Java[5], C#[6], and Javascript[7]. [1] https://en.wikipedia.org/wiki/K_(programming_language) [2] https://en.wikipedia.org/wiki/Q_(programm…
Thank you. Now when I'm thinking some code I have to untangle is bad, I'd always remind myself "at least it's not K-inspired C"...
Re: An oral history of Bank Python
#15Having worked in an investment bank before this brought some flashbacks :)
Re: An oral history of Bank Python
#16Sounds like it could be JPM's 'Athena' platform? context: https://www.techrepublic.com/article/jpmorgans-athena-has-35...
Certainly does. Some discussion here from a few years back:
Re: An oral history of Bank Python
#17Compared to one major IB bank Python system, this is all extremely clean and neat. Consider a Python API that is a thin wrapper on COM calls intended to be used from Excel. Want to request some data? Fill in a 2D virtual Excel table. Want to pull some data? Query it and parse a text-dump of a table excerpt (remembering to parse #NA! Etc as nans). Want to automate a job? Enter it as a new row to a global spreadsheet.…
Or they could use instead use CSVs. What could possibly go wrong?
Everything is fine, as long as no Americans come and write 1,000 where obviously they should have written 1000 or 1'000. /s
Re: An oral history of Bank Python
#18Reminds me of what we used at the ATLAS experiment at CERN* . Python was tightly integrated with the application framework, Athena (which I just realize has the same name as JPM's Python framework!). You could use it as a job description language, and you would compose computation steps from classes you could write in C++. I think there was a separate `athena` executable that was just python with some packages pre-loaded. Because of all the binary modules, but even more so because of the minor syntax changes, the transition to Python 3 was really a problem (I hope they did it by now :-D).
There was also a bespoke time-span database. You could store keys and values in there, but every data point had a start and end time. Then you could query what the values were between certain times, or run numbers (operational periods). We used it for example to store what configuration the detector was using when a certain dataset has been recorded.
(* I've been out for a couple of years so I don't know what they use now, but I imagine it hasn't changed much.)
Re: An oral history of Bank Python
#19I'm seeing a lot of people speculating about which bank this might be; I think the point is that it's all of them. I could loosely describe a previous job as implementing Morgan Stanley's Walpole and integrating more source code management into Minerva (even though that system wasn't actually Python-based).
Having a global view on everything is large banks' value-add, it's why they haven't been outcompeted by their more nimble competitors. Being able to calculate the risk of the whole bank isn't just a cool feature, it's the core value proposition of this platform.
Being able to just upload your code and run it is really cool, and if you squint it looks a bit like what the outside world is trying to set up with serverless/lambda-style platforms - just write a function, submit it, and there, it's running. (But it's worth remembering that Python is not a typical programming language; python build, dependency and deployment management is exceptionally awful in every respect, this isn't as big a pain point in other languages). Obviously there's a tension between this and having good version control, diffs, easy rollbacks etc. - but because Minevera is already designed to do all that for data (because you need that kind of functionality for modifications to your bonds or whatever), doing it this way strikes a much better compromise than something like editing PHP files directly on your live server.
What this article calls data-first design has a lot in common with functional programming. I hope that as the outside world adopts more functional programming and non-relational datastores, Minerva-style programming will get more popular. It really is a much better way to write code in many ways. The difficulty of integrating with outside libraries is a shame though.
Re: An oral history of Bank Python
#20I can see the benefits of this collection of tools within an all-in-one monolith. Ease of deployment is a big benefit. I can also see the costs. As a stack its probably better in some ways than how a lot of other businesses operate as well as worse. There's probably a lot both ways. The mainframe mindset might be a factor here as well. The giant mainframe where all the magic happens is still a thing to behold and thi…
> The part about programming languages frequently not having tables is interesting. The closest as mentioned is the hash, but you lose so much in that abstraction eg the relational aspects. The counter argument then becomes the obvious: why aren't you using a database library, or in a pinch, sqlite? Rightly so. Why would you add relational tables to python rather than have a generic python database spec or a collection of database connector libraries. Databases are separate and large projects in themselves.
The separate datastore is the problem to be solved here - databases, especially relational databases, are extremely poorly integrated into programming languages and this makes it really painful to develop anything that uses them. You can just about use them as a place to dump serialized data to and from (not suitable for large systems because they're not properly distributed), but if you actually want to operate on data you need it to be in memory where you're running the code and you want it to be tightly integrated with your language and IDE and so on.
(It's not even the main benefit, but just as an example of that kind of integration, when you're querying large datasets Minerva works a bit like Hadoop in that it will ship your code to where the data is and run it there)