Jane Street and the OCaml Compiler (2018) [video]
janestreet.com
Jane Street and the OCaml Compiler (2018) [video]
1–10 of 119 posts
Re: Jane Street and the OCaml Compiler (2018) [video]
#2i asked this question at my Jane Street interview, but never got a real answer: how does a market making latency arb firm handle GC pauses that are inherent with any GC language? OCaml is better in this regard than, say, the JVM, but it still seems problematic. I know they wrote some FPGA compiler stuff with OCaml, but I can't imagine that all of their execution is running through FPGAs. Even if it's possible, it seems like an uphill battle; something like Rust would probably be easier to deal with (at least for their realtime code).
Re: Jane Street and the OCaml Compiler (2018) [video]
#3i asked this question at my Jane Street interview, but never got a real answer: how does a market making latency arb firm handle GC pauses that are inherent with any GC language? OCaml is better in this regard than, say, the JVM, but it still seems problematic. I know they wrote some FPGA compiler stuff with OCaml, but I can't imagine that all of their execution is running through FPGAs. Even if it's possible, it see…
I don't know what Jane Street does, and can't speak for them. That being said, I've heard of some firms using the JVM that just tune things such that the garbage collector will never trigger, and then they restart the VM every night.
Unfortunately I don't have a citation for that. I think I've seen it talked about before here on HN before.
Re: Jane Street and the OCaml Compiler (2018) [video]
#4i asked this question at my Jane Street interview, but never got a real answer: how does a market making latency arb firm handle GC pauses that are inherent with any GC language? OCaml is better in this regard than, say, the JVM, but it still seems problematic. I know they wrote some FPGA compiler stuff with OCaml, but I can't imagine that all of their execution is running through FPGAs. Even if it's possible, it see…
This is a really fascinating question. When I chatted with some of the Jane Street crew at NeurIPS they were adamant in the benefits of OCaml's Hindley-Milner type system.
I didn't ask, but am curious, how that compares to the type guarantees of Rust? Would moving to Rust cause them to lose that advantage of compile time error catching? I've never written a line of Rust (hopefully that changes soon) so I don't know, but am certainly interested.
Re: Jane Street and the OCaml Compiler (2018) [video]
#5i asked this question at my Jane Street interview, but never got a real answer: how does a market making latency arb firm handle GC pauses that are inherent with any GC language? OCaml is better in this regard than, say, the JVM, but it still seems problematic. I know they wrote some FPGA compiler stuff with OCaml, but I can't imagine that all of their execution is running through FPGAs. Even if it's possible, it see…
I don't know what Jane Street does, and can't speak for them. That being said, I've heard of some firms using the JVM that just tune things such that the garbage collector will never trigger, and then they restart the VM every night. Unfortunately I don't have a citation for that. I think I've seen it talked about before here on HN before.
I'm not entirely clear on the use case, but Disruptor by LMAX Exchange is for the JVM and seems very concerned with latency. (https://github.com/LMAX-Exchange/disruptor/wiki/Performance-...)
Re: Jane Street and the OCaml Compiler (2018) [video]
#6i asked this question at my Jane Street interview, but never got a real answer: how does a market making latency arb firm handle GC pauses that are inherent with any GC language? OCaml is better in this regard than, say, the JVM, but it still seems problematic. I know they wrote some FPGA compiler stuff with OCaml, but I can't imagine that all of their execution is running through FPGAs. Even if it's possible, it see…
Things to consider:
The company is >20 years old.
OCaml is a general purpose language that provides an equilibrium between the avoidance of bugs introduced by state (like all functional languages), speed, and polymorphic type inference. At the time of adoption, the other choices were Haskell (too academic, not practical), Erlang (no type inference, not suited for large code bases with complex business logic), and lisp (too slow, loose/optional type system). The last time I checked, OCaml was third only to C and C++ in terms of speed. It is also important to consider how intellectually stimulating it is to write OCaml. If you can achieve the three things mentioned at the top of this paragraph while also creating a brand of gravitas and intellect that attracts top-tier talent, of course you would choose OCaml.
Would a new, uninitiated market maker write something in OCaml? Unlikely, as they would probably use C++, Rust, or Scala with a 1TB heap and GC disabled. Ignoring the learning curve and time/dollar constraints of starting a hedge fund, I would choose OCaml over the three mentioned.
[1]https://en.m.wikipedia.org/wiki/Hindley%E2%80%93Milner_type_...
[2]https://courses.cs.washington.edu/courses/cse341/04wi/lectur...
Re: Jane Street and the OCaml Compiler (2018) [video]
#7i asked this question at my Jane Street interview, but never got a real answer: how does a market making latency arb firm handle GC pauses that are inherent with any GC language? OCaml is better in this regard than, say, the JVM, but it still seems problematic. I know they wrote some FPGA compiler stuff with OCaml, but I can't imagine that all of their execution is running through FPGAs. Even if it's possible, it see…
In Elixir/Erlang there is no global pausing for garbage collection, instead each process is individual garbage collected. And I remember hearing about a strategy that allows you to avoid it entirely by ensuring your process's lifetime is short enough (or heap size big enough) that it basically never happens despite the BEAM (Erlang VM) running with garbage collection.
Edit: my point is that it is likely avoidable through certain coding practices in garbage collected environments.
Re: Jane Street and the OCaml Compiler (2018) [video]
#8Earlier quoted context omitted.
I don't know what Jane Street does, and can't speak for them. That being said, I've heard of some firms using the JVM that just tune things such that the garbage collector will never trigger, and then they restart the VM every night. Unfortunately I don't have a citation for that. I think I've seen it talked about before here on HN before.
I'm not entirely clear on the use case, but Disruptor by LMAX Exchange is for the JVM and seems very concerned with latency. ( https://github.com/LMAX-Exchange/disruptor/wiki/Performance-... )
Here's the backstory, worth a good read in general. https://martinfowler.com/articles/lmax.html
I suspect at least one use case has gone away since the Fowler wrote that: wanting to saturate the write channel into a database in general. Consistent hashing document DBs like DynamoDB, Cassandra, etc. are basically infinitely scalable for writes. It's not clear to me if LMAX still makes sense if you want to assemble the db writes into a strongly ordered stream.
Re: Jane Street and the OCaml Compiler (2018) [video]
#9i asked this question at my Jane Street interview, but never got a real answer: how does a market making latency arb firm handle GC pauses that are inherent with any GC language? OCaml is better in this regard than, say, the JVM, but it still seems problematic. I know they wrote some FPGA compiler stuff with OCaml, but I can't imagine that all of their execution is running through FPGAs. Even if it's possible, it see…
Things to consider: The company is >20 years old. OCaml is a general purpose language that provides an equilibrium between the avoidance of bugs introduced by state (like all functional languages), speed, and polymorphic type inference. At the time of adoption, the other choices were Haskell (too academic, not practical), Erlang (no type inference, not suited for large code bases with complex business logic), and lis…
Why C++ now but not then, 20+ years ago? And would F# be suitable for the task? Im genuinely interested
Re: Jane Street and the OCaml Compiler (2018) [video]
#10Earlier quoted context omitted.
Things to consider: The company is >20 years old. OCaml is a general purpose language that provides an equilibrium between the avoidance of bugs introduced by state (like all functional languages), speed, and polymorphic type inference. At the time of adoption, the other choices were Haskell (too academic, not practical), Erlang (no type inference, not suited for large code bases with complex business logic), and lis…
Why C++ now but not then, 20+ years ago? And would F# be suitable for the task? Im genuinely interested
Presumably Jane Street didn’t choose C++ because they wanted to reduce bugs introduced by the preservation of state; the killer of prop shops. F# was developed six years after their founding, hence too young, and more importantly, a Microsoft-owned clone of OCaml. I don’t think it even ran on Linux before 2015. Today, F# might just suit the job, assuming you are open to being locked into the .NET family. An interesting idea to say the least.
Why C++ now? Still the fastest and tons of quants and highly skilled programmers know it. When you consider the correlation between C++ developers’ technical acumen and quantitative skills, coupled with the maturity and increasing convenience of the ecosystem, it makes sense.