111–120 of 146 posts
Earlier quoted context omitted.
https://www.youtube.com/watch?v=v1CmGbOGb2I
that seems to be a presentation about language features. I'm mostly interested in the business reasons for using the language within what jane street does, and how the language offers a competitive advantage and why it is "good enough" for the highly competitive HFT landscape they work in.
Tangentially related. But i really love the podcast from the janestreet people: signals and threads.
Can I ask an office hours type question? I worked on a very similar (if not identical lol) project at a job once upon a time and the biggest problem I had (and one that I never really solved well) was recovering call stacks from trace data. I effectively ended up using DWARF and just simulating execution and keeping a call stack in the decoder. This mostly worked fine for small and simple programs, but I ran into SO…
- I imagine the extra memory bandwidth of newer parts doesn't hurt. The example traces were taken on server-class Ice Lake machines. They just don't overflow for our typical workloads.
- We found the specific IPT configuration matters a lot. Turning off return compression is more liable to result in overflows. We allow varying this in magic-trace via the `-timing-resolution` parameter, more detail available in the wiki. We don't typically see overflows under the default configuration even on Broadwell server-class parts.
- Clark spent a week on an Intel NUC (mobile Tiger Lake part) toiling away on decode error recovery. For the most part, the data lost are uninteresting branches, and you only need one of the call in / return out of a frame to survive the decode error to be able to construct a frame for it.
We also considered the periodic stack sampling approach for error recovery, but ended up not implementing it since the decode error recovery we implemented ended up being robust enough in practice.
We ended up having more trouble with runtimes that mess with the stack pointer directly. (The kernel does this for the retpoline Spectre mitigation! But perf is smart and rewrites that part of the instruction stream into a jump for us.) There's code in magic-trace to special-case OCaml exceptions, for instance, and it's likely similar code is necessary for some other runtimes too (we have an open issue for Go's coroutine switching).
Earlier quoted context omitted.
Of course they use it for trading and everything around for many years.
cool, so how about the OP shares the details of how they use it for trading?
https://signalsandthreads.com/
> Listen in on Jane Street’s Ron Minsky as he has conversations with engineers working on everything from clock synchronization to reliable multicast, build systems to reconfigurable hardware. Get a peek at how Jane Street approaches problems, and how those ideas relate to tech more broadly.
Earlier quoted context omitted.
If by "here" you mean "at academic conferences", then I'd ask you why they bother to actually do primary research and publish it if they have no interest in sharing knowledge. If all they wanted was to hire people, they... would. You don't have to sponsor a conference to attend or hire from that conference. And you especially also don't need to sponsor additional workshops, or carbon neutrality initiatives, or anythi…
because they are hoping to run into interesting people to hire. why would they share knowledge otherwise? if they are interested in doing that, why not share all the details of their tech stack and what they are doing exactly in the market? it's very hard to hire, especially with FAANGs who compete with them for talent.
Because they believe in the value of science?
They don't need to publish to compete with top-tier public companies. I don't know of any other trading firm that contributes to open-source development, or to a language infrastructure, or to academic advancement in the way and to the extent that Jane Street do. Most companies keep everything proprietary and highly secret.
But JS chooses to publish. And it's not like that's an easy task that you can just do for fun on a whim; it takes a long time to put together a good paper. They also regularly collaborate with people in academia on long-term projects and evaluations.
I understand the perspective you're suggesting, but I genuinely believe it is wrong, and I also believe that you do not have any evidence to back it up. I think their public contributions speak for themselves, but I've also met some of their more academically inclined engineers (including their CTO), and they come from an academic background and seem to genuinely believe in academic publishing as a goal in itself. It's not totally crazy that there exists one such company out there. (There are actually a couple, but not terribly many, and the others are not relevant in the present discussion.)
You should make it clearer in the magic trace Readme that the UI service fork used for viewing and analyzing the traces is also available should one want to deploy it locally or in a situation without Internet access: https://github.com/janestreet/perfetto/ It is mentioned in the documentation but for anyone quickly skimming and expecting a SaaS pricing model underneath (I think many do now), that isn't obvious. From…
I've recently stumbled on Google's Perfetto in the last few months. Very very nice UI. I've been using it with viztracer for Python.
IPT - so I'll need some kind of a recent *Lake CPU to have this support? And the tool won't work on EPYC, I guess?
IPT - so I'll need some kind of a recent *Lake CPU to have this support? And the tool won't work on EPYC, I guess?