Live data from Hacker News

C++ patterns for low-latency applications including high-frequency trading

arxiv.org

181–190 of 240 posts

Re: C++ patterns for low-latency applications including high-frequency trading

#181
post #3

I've got an implementation of a stock exchange that uses the LMAX disruptor pattern in C++ https://github.com/sneilan/stock-exchange And a basic implementation of the LMAX disruptor as a couple C++ files https://github.com/sneilan/lmax-disruptor-tutorial I've been looking to rebuild this in rust however. I reached the point where I implemented my own websocket protocol, authentication system, SSL etc. Then I realized…

I'll have to take a look at the code. Maybe I can integrate it into https://github.com/rburkholder/trade-frame

Re: C++ patterns for low-latency applications including high-frequency trading

#182

Earlier quoted context omitted.

You don't need to shut down processes on the server. All you have to do is isolate CPU cores and move your workloads onto those cores. That's been a common practice in low latency networking for decades.

I'm not in HFT, but I wouldn't expect that to be enough. Not only do you want to isolate cores, you want to isolate any shared cache between cores. You do not want your critical data ejected from the cache because a different core sharing the cache has decided it needs that cache. Which of course starts with knowing exactly what CPU you are using since different ones have different cache layouts. You also don't want…

Correct. I was just pointing out to OP that moving processes is not worthwhile and isolation is how you'd do it

Re: C++ patterns for low-latency applications including high-frequency trading

#183

Earlier quoted context omitted.

> These HN individuals are not domain educated and are spouting highly uninformed nonsense. Yeah, this is highly frustrating particularly for people like me who don't know anything about the domain i.e. HFT/Trading and would like to know more. Can you recommend some good introductory books/resources ?

I might be biased for some almost personal reasons, however I think there is one book that anyone serious about the field should just read, period. “Trades, Quotes and Prices” Authors: Jean-Philippe Bouchaud, Capital Fund Management, Paris, Julius Bonart, University College London, Jonathan Donier, Capital Fund Management, Martin Gould, CFM - Imperial Institute of Quantitative Finance. I’m a practitioner and this boo…

Thank You.

Just FYI for others; the full name of the book is "Trades, Quotes and Prices : Financial Markets Under the Microscope".

Looks like a very comprehensive book though somewhat advanced for me at my current level. Do you have a more beginner level book/resource recommendation to go with this where from i can get an overview and familiarize myself with the jargon?

Re: C++ patterns for low-latency applications including high-frequency trading

#184

Earlier quoted context omitted.

I briefly looked over your stock exchange code: - For memory management, consider switching to std::shared_ptr. It won't slow anything down and will put that concern to rest entirely. - For sockets, there are FOSS libraries that will outperform your code and save you a ton of headaches dealing with caveats and annoyances. For example, your looping through FD_ISSET is slower than e.g. epoll or kqueue. - For dependenci…

I did not know std::shared_ptr would not slow things down. I've learned something new today! :) Yes, I agree, epoll is a lot better than FD_ISSET. Maybe I can keep moving with my C++ code but do people still trust C++ projects anymore? My ideal use case is a hobbyist who wants a toy stock exchange to run directly in AWS. I felt that C++ has a lot of bad publicity and if I want anyone to trust/try my code I would have…

std::shared_ptr definitely slows things down. It's non-intrusive therefore requires a memory indirection.

Re: C++ patterns for low-latency applications including high-frequency trading

#185

Earlier quoted context omitted.

When I did low latency everyone was offloading TCP to dedicated hardware. They would shut down every single process on the server and bind the trading trading app to the CPUs during trading hours to ensure nothing interrupted. Electrons travel slower than light so they would rent server space at the exchange so they had direct access to the exchange network and didn't have to transverse miles of cables to send their…

Any good books/resources you can recommend to learn about the above architectures/techniques?

Some years ago I wrote a gist about HFT/HPC systems patterns (versus OPs C++ patterns) applied to dockerized Redis. Might be dated, but touches on core isolation/pinning, numa/cgroups, kernel bypass, with some links to go deeper. Nowadays I do it with Kubernetes and Nomad facilities, but same basic ideas:

https://gist.github.com/neomantra/3c9b89887d19be6fa5708bf401...

Re: C++ patterns for low-latency applications including high-frequency trading

#186

Earlier quoted context omitted.

On the software side I don't think HFT is as special a space as this paper makes it out to be.[1] Each year at cppcon there's another half-dozen talks going in depth on different elements of performance that cover more ground collectively than any single paper will. Similarly, there's an immense amount of formal literature and textbooks out of the game development space that can be very useful to newcomers looking fo…

Latency isn't even as important in HFT as people claim. What's most important is deterministically staying into a reasonable enveloppe (even at the 99.9 percentile) to satisfy real-time requirements and not fall behind. When it's really important, it's implemented in FPGA or with an ASIC.

This is close to the question I asked above, comparing to audio. If you're processing 32 samples at a time at 48kHz, you need to process a frame every 667us. If you only hit the deadline 99.9% of the time, that's a glitch more than once per second. You'd need a bunch more nines for an acceptable commercial product, so most audio devs just treat it as a hard realtime constraint.

I think the main split is whether you care more about average latency or worst-case latency (or some # of nines percentile).

Re: C++ patterns for low-latency applications including high-frequency trading

#187

Earlier quoted context omitted.

When I did low latency everyone was offloading TCP to dedicated hardware. They would shut down every single process on the server and bind the trading trading app to the CPUs during trading hours to ensure nothing interrupted. Electrons travel slower than light so they would rent server space at the exchange so they had direct access to the exchange network and didn't have to transverse miles of cables to send their…

Any good books/resources you can recommend to learn about the above architectures/techniques?

A few episodes of Signals and Threads, a podcast from Jane Street, go into parts of it.

Re: C++ patterns for low-latency applications including high-frequency trading

#188
post #178

Earlier quoted context omitted.

Audio has a lot of buffering behaviour that you wouldn't generally see in event-reactive HFT. Think of all the plugins that you know of that have non-zero latency, compressors with 'lookahead' etc. There are maybe some similarities where the logic is more complex (loop unrolling, SIMD and so on) but I feel like plugins are generally optimizing for throughput (CPU usage) and quality (oversampling etc) rather than pure…

I guess the difference I'm interested in is whether HFT tends to be "realtime", in the sense that there's a hard deadline that you need to hit every time. Put another way, audio is operating on a much longer timescale, but cares a lot about worst-case latency (as well as throughput and quality). Is that true of HFT also, or are they more concerned with average latency? If computing a trade takes too long, can they ju…

I can't go into too much detail (I currently work at an HFT, but not on the HFT bits), but some of the FPGAs make decisions without even seeing a entire packet coming in on the wire. Latencies are more often measured in nanos than micros from what I've seen lately.

Re: C++ patterns for low-latency applications including high-frequency trading

#189
post #114

Earlier quoted context omitted.

As someone who does quant trading professionally and game development as a hobby, they both are performance sensitive, but they emphasize different kinds of performance. Trading is about minimizing latency while video games are about maximizing bandwidth. Video games try to cram as much work as possible within about 16 milliseconds whereas for most trading algorithms 16 milliseconds is too slow to do anything, you wa…

I'm curious how HFT relates to pro audio programming. The timescale is close to gaming (usually It's not hard real-time like you're going to crash your car, but if you miss your deadline it causes an unacceptable and audible glitch. I've always been a bit surprised that Jane Street uses OCaml. I know they've put a lot of attention into the GC, but it still seems fundamentally indeterminate in a way that would make mo…

As somebody who has done both, there's certainly a lot of similarities in how you write software.

I'm talking about real-time audio, though. There's also just audio playback (Pro Tools, e.g.) which tolerates much higher latency, but enables higher audio fidelity due to larger buffer sizes (e.g. in compressors to enable look-ahead) as well as more clever algorithms (which are often prone to latency spikes, such as a "standard" FFT, if you want a very simple example).

Both, finance and real-time audio, differ in their time scales, but more in the breadth of their time scales: real-time audio is often between 1–50 ms (yeah, we try to keep it below 10 ms, but that's tough to do in a complex setup. I've seen artists often compromise and increase buffer sizes (== latency) to accommodate for their complex virtual instruments and filters), finance in ns–ms (it's not HFT anymore at a certain point, sure, but those HFT architectures also have a layered response, with FPGAs being quick but simple and more complex strategies having higher latency; also, if you're e.g. asked for a quote on a bond, it's not a shame to take 500 ms to think about a price, because those trades are bigger but slower).

You also mentioned the consequences of missing a deadline. Well, indeed, I've seen this variance to be less of an issue in finance (we're only caring about xx µs, so I hesitate to call myself a HFT guy) than in real-time audio. A click in a live set is a no-go. A crash of your DAW is fatal, that's it with your live set. Nobody dies, but people will be very upset. I've seen real-time audio being obsessed with being reliable and keeping the variance in latency really small.

In finance, I've seen it more that people kind of accepted that variance was huge because there's some hidden allocations in a system. Hoping that those stop once the system is warmed up or that they rarely occur on a critical trade. As a side note, in finance I've even heard this well-known HFT firm: "Well, it's not the end of the world if your process crashes. If you can restart quick enough."

It's also that in finance, your data is more heterogenous: Real-time audio has audio (isochronous, homogenous buffer of floats), MIDI (asynchronous, formerly small events), parameter changes (asynchronous, often just a float) and config changes (changes the topology of your audio processing graph; enjoy to transition from A to B without causing an audible glitch). Finance has processing graphs that only process asynchronous data (heterogenous market data, telling you what's on the order book of an exchange; trade data for which trades you performed with your orders and quotes; symbol data when new securities appear or change (historically only with the start of the trading day); pre-computed steering data for algorithms and/or data from third-party data sources).

Thus, the processing graphs of both differ in the data that's transferred as well as in the structure and domain. In finance, there is no regular tick-based processing. You have to accommodate a very varying amount of data per time period, whereas in audio, it's a very regular pace and the amount of data is often the same (± activations of voices in instruments).

Real-time audio has plugins, though. And those, from experience, do whatever you can imagine. Plugins are great, from a creative perspective, but the craftsmanship is of very varying degree. I bet I'll even find a plugin that draws the UI from the real-time audio thread.

Any other experiences from the real-time realm?

Re: C++ patterns for low-latency applications including high-frequency trading

#190

Earlier quoted context omitted.

It's not easy to get data structures like this right in C++. There are a couple of problems with your implementation of the queue. Memory accesses can be reordered by both the compiler and the CPU, so you should use std::atomic for your producer and consumer positions to get the barriers described in the original LMAX Disruptor paper. In the get method, you're returning a pointer to the element within the queue after…

>> In the get method, you're returning a pointer to the element within the queue after bumping the consumer position (which frees the slot for the producer), so it can get overwritten while the user is accessing it. And then your producer and consumer positions will most likely end up in the same cache line, leading to false sharing. I did not realize this. Thank you so much for pointing this out. I'm going to take a…

I have absolutely no idea how this works in Java, but in C++, there are a few reasons you need std::atomic here:

1. You need to make sure that modifying the producer/consumer position is actually atomic. This may end up being the same instruction that the compiler would use for modifying a non-atomic variable, but that will depend on your target architecture and the size of the data type. Without std::atomic, it may also generate multiple instructions to implement that load/store or use an instruction which is non-atomic at the CPU level. See [1] for more information.

2. You're using positions for synchronization between the producer and consumer. When incrementing the reader position, you're basically freeing a slot for the producer, which means that you need to make sure all reads happen before you do it. When incrementing the producer position, you're indicating that the slot is ready to be consumed, so you need to make sure that all the stores to that slot happen before that. Things may go wrong here due to reordering by the compiler or by the CPU [2], so you need to instruct both that a certain memory ordering is required here. Reordering by the compiler can be prevented using a compiler-level memory barrier - asm volatile("" ::: "memory"). Depending on your CPU architecture, you may or may not need to add a memory barrier instruction as well to prevent reordering by the CPU at runtime. The good news is that std::atomic does all that for you if you pick the right memory ordering, and by default, it uses the strongest one (sequentially-consistent ordering). I think in this particular case you could relax the constraints a bit and use memory_order_acquire on the consumer side and memory_order_release on the producer side [3].

[1] https://preshing.com/20130618/atomic-vs-non-atomic-operation...

[2] https://en.wikipedia.org/wiki/Memory_ordering

[3] https://en.cppreference.com/w/cpp/atomic/memory_order

Post reply on HN