Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An oral history of Bank Python

calpaterson.com

Post reply on HN
Print view View original

181–190 of 333 posts

Re: An oral history of Bank Python

#181

Post by twic »

odiroot wrote: post #98
urban_winter wrote: post #53

BAML Quartz was conceived by a bunch of front-office quants who had not the first idea about the software needs of a big bank beyond the front office. There was an arrogant assumption that front office software is obviously the most complicated/difficult variety of software within a bank and therefore any system designed with front office requirements at the forefront would, of course, be perfect for universal use. T…

Can you have a career in finance as an engineer with Python and without C/C++ (professional) experience? Your post really made think it, it's an attractive area to work in.

Yes. Tons of it is in Java.

Re: An oral history of Bank Python

#182

Post by twic »

cyberpunk wrote: post #135

Both frontarena and murex use python as their “vb” kind of language. If you thought your deployment pipelines were weird, ours have included putting entire python apps into single strings and inserting them to an oracle db, where a fat windows client selects them and runs them on a windows python interpreter … via citrix… :/

This reminds me of an e-commerce system which stored data in a mixture of Oracle, and text files on the local disk. We handled backups by loading the text files into blob columns in the database, and then just backing up the database.

Re: An oral history of Bank Python

#183

Post by jetbooster »

dwohnitmok wrote: post #170
qorrect wrote: post #161

> You can achieve a awful lot with Excel: more, even, than some programmers can achieve without it. I've heard this a lot, but have never really used Excel. What can it do that a programmer can't ?

By virtue of Turing completeness there's nothing you can do in Excel that you can't do in a program. It's all a matter of speed. Having seen Excel wizards work their magic before, the dizzying ways they can slice and dice their data with the help of a combination of GUI affordances, formulas, and hot keys is truly astounding. Often times a person could build out a full set of data and charts in half an hour that migh…

And I think here-in lies the trap of excel.

You've built this brilliant report because you're an excel wiz, but because of that you've gotten someone up the chain's attention and you need to do it every day/multiple times per day, and automating out all your shortcuts, hotkey and ui clicks with macros becomes a horrifying cludge that had you invested in something more automation oriented earlier would produce more resilient/repeatable solutions.

Re: An oral history of Bank Python

#184

Post by nonesuchluck »

dwohnitmok wrote: post #170
qorrect wrote: post #161

> You can achieve a awful lot with Excel: more, even, than some programmers can achieve without it. I've heard this a lot, but have never really used Excel. What can it do that a programmer can't ?

By virtue of Turing completeness there's nothing you can do in Excel that you can't do in a program. It's all a matter of speed. Having seen Excel wizards work their magic before, the dizzying ways they can slice and dice their data with the help of a combination of GUI affordances, formulas, and hot keys is truly astounding. Often times a person could build out a full set of data and charts in half an hour that migh…

Yep. The peculiarities that make this Excel workflow unreasonably effective are pretty easy to identify:

1. Tabular data. There's some tricks with named ranges etc, but for the most part your entire application state is spread out before you, scrollable in every direction. It's just tables, and clicking into a cell highlights relationships (data dependency) between cells.

2. Visible data, hidden code. =macros are hidden behind their result; the most obvious thing is to treat them as tiny black boxes, applying a single data transformation (or small set of logically related transforms), and immediately see the result applied to your data set. This is a tiny bit like functional programming, and a tiny bit like Pandas or Spark (immutable data, lazy evaluations). Except unlike those worlds, Excel pushes the data front and center, not the code.

And prob a bunch more I'm forgetting. It doesn't even feel like programming, more like building data pipelines in Unix or something. Except you can easily preview the data at each step in the pipeline. What I really want is Excel or Siag, but with Python and SQLite and a nice spreadsheet UI.

Re: An oral history of Bank Python

#185

Post by denimnerd42 »

mtrovo wrote: post #152

> I've mentioned that programmers are far too dismissive of MS Excel. You can achieve a awful lot with Excel: more, even, than some programmers can achieve without it This is one of the most underrated topics in tech imho. Spreadsheet is probably the pinnacle of how tech could be easily approachable by non tech people, in the "bike for the mind" sense. We came a long way down hill from there when you need an speciali…

problem as described to me is that excel starts being used for regulated processes and it's not well auditable, access controlled, changed controlled, tracked, etc etc. Then people need to implement the exact same process across departments and they're all using a separate excel sheet and they all submit different numbers. becomes a huge mess and so much more complicated and expensive systems become commissioned.

Re: An oral history of Bank Python

#186

Post by mst »

leprechaun1066 wrote: post #82
Thorrez wrote: post #4

Doesn't seem that strange compared to K[1] or Q[2], which are used by Wall Street banks. K encourages you to use single-letter variables and bunch your code up as tight as possible into long lines. Here's an example: [3]. Interestingly, their Github repo has some K-inspired C[4], Java[5], C#[6], and Javascript[7]. [1] https://en.wikipedia.org/wiki/K_(programming_language) [2] https://en.wikipedia.org/wiki/Q_(programm…

The big banks don't write code in K4 though, managers generally encourage people not to write code in it due to the difficulty of finding developers who are fluent in it. They all use q and q is very wordy and highly readable if you speak english. It's mostly just developer defined functions which are compositions of the keywords of which there are not many: https://code.kx.com/q/ref/#keywords Most of the code you wo…

Do you have any suggestions for q code to look at? Every time I try array languages I bounce off the ubercompact and I feel like there's actually a chance I could learn something more q like.

Re: An oral history of Bank Python

#187

Post by mst »

zzzeek wrote: post #121
rich_sasha wrote: post #3

Compared to one major IB bank Python system, this is all extremely clean and neat. Consider a Python API that is a thin wrapper on COM calls intended to be used from Excel. Want to request some data? Fill in a 2D virtual Excel table. Want to pull some data? Query it and parse a text-dump of a table excerpt (remembering to parse #NA! Etc as nans). Want to automate a job? Enter it as a new row to a global spreadsheet.…

this is the approach I'm familiar with, but back when I did it in Perl (yes Win32 perl with COM bindings) and Java (we wrote native Java plugins to do COM to Excel). all the important stuff is Excel VBA code that they spent years developing and can never replace, so any front end type of thing had to somehow get back to the Excel models. We eventually did rewrite the Excel models in Java, released something, and then…

I remember back in 2000 converting the windows line of business app team at $ISP (I was mostly on the provisioning automation side) over to using a COM component called JendaRex which wrapped the perl VM just to expose the regexp engine.

This basically came about after the Nth time they asked for regexp help and I had a trivial solution that didn't work in whatever native implementation they were using and I basically gave them a choice between JendaRex and "not having me debugging their regexps anymore".

They unanimously chose JendaRex and everybody ended up happier as a result.

Re: An oral history of Bank Python

#188

Post by joshu »

lmm wrote: post #19

I'm seeing a lot of people speculating about which bank this might be; I think the point is that it's all of them. I could loosely describe a previous job as implementing Morgan Stanley's Walpole and integrating more source code management into Minerva (even though that system wasn't actually Python-based). Having a global view on everything is large banks' value-add, it's why they haven't been outcompeted by their m…

it's been 15 years and i'm still a bit traumatized by aurora

Re: An oral history of Bank Python

#189

Post by arthurcolle »

arthurcolle wrote: post #144
seanhunter wrote: post #76

Earlier quoted context omitted.

I know Kirat really well. Fun fact, one two-week dev cycle we had 667 distinct developers commit to the secdb code base which Kirat's boss described to me as "The number of the beast.... plus Kirat" Second fun thing. Kirat was advocating for lisp for secdb for a long time and used to rag on me for liking python when it's so slow.

Had a lot of fun pulling things out of MetaDir and recreating what seemed like "early history" of SecDB when I was there, was a lot of fun :D

the spirit of dubno lives on...

Re: An oral history of Bank Python

#190

Post by sam0x17 »

mtrovo wrote: post #152

> I've mentioned that programmers are far too dismissive of MS Excel. You can achieve a awful lot with Excel: more, even, than some programmers can achieve without it This is one of the most underrated topics in tech imho. Spreadsheet is probably the pinnacle of how tech could be easily approachable by non tech people, in the "bike for the mind" sense. We came a long way down hill from there when you need an speciali…

Very true. I often prototype algorithms and things in google sheets. One time I had backpropagation working in there, with a little button to process the next "row" of training samples.
Post reply on HN