Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C++ patterns for low-latency applications including high-frequency trading

arxiv.org

Post reply on HN
Print view View original

211–220 of 240 posts

Re: C++ patterns for low-latency applications including high-frequency trading

#211

Post by cout »

HolyLampshade wrote: post #177
vinay_ys wrote: post #175

Earlier quoted context omitted.

Isn't there challenges with slippage and managing the volumes while exiting? And isn't speed also about processing the data feed as fast as possible to time the exit decisions accurately?

Absolutely to both questions, with different answers depending on what style of strategy you’re running. The market mechanics trades tend to have no recoverability if you miss the opportunity, so you’re often trading out in error and it’s a matter of trying to stem the loss on a position that you do not have an opinionated value signal on. And there’s definitely an angle to inbound processing speed for both styles of…

This is true -- in the early 00's there was hardly any competition and we could take out both sides of a price cross with 100ms latency. Even after colocation we could still be competitive with over 4ms latency (plus the network). Trading technology has come a long way in 20 years.

Re: C++ patterns for low-latency applications including high-frequency trading

#212

Post by mgaunard »

atomicnumber3 wrote: post #197
mgaunard wrote: post #147

Earlier quoted context omitted.

Even if the low-latency logic moves to the FPGA, you still need your slow path to be reasonably fast, say about 100us, and absolutely never more than 1ms. To my knowledge Python is not suitable, though I know some players embed scripting languages that are.

It depends entirely on what you've got on the FPGA and what's slow-path. My firm had order entry done exclusively by the FPGA and the non-FPGA code would have orders in the FPGA for sometimes tens of seconds before they would fire. The non-FPGA code was still C++, but the kind of everyday C++ where nobody has been wringing all the water out of it for speed gains. If we rewrote it all, it'd probably be python. But tha…

Your hardware is configured by your software. You pre-arm transactions to be released when a trigger occurs.

Releasing the transaction when the trigger occurs is the fast path, and what the hardware does. The slow path is what actually decides what should be armed and with what trigger.

If your software is not keeping up with information coming in, either you let that happen and trigger on stale data, or you automatically invalidate stale triggers and therefore never trigger.

100us is short enough to keep up with most markets, though obviously not fast enough to be competitive in the trigger to release loop.

But it's mostly an interesting magnitude to consider when comparing technologies; Python can't really do that scale. In practice a well-designed system in C++ doesn't struggle at all to stay well below that figure.

Re: C++ patterns for low-latency applications including high-frequency trading

#213

Post by ketanhwr »

kangda123 wrote: post #206
vineyardlabs wrote: post #193

Earlier quoted context omitted.

Tangentially, how do you like working for an HFT? I'm a low level software / FPGA developer and have thought about going into the HFT space. Is the work life balance as terrible as people say?

Not OP but it varies wildly between companies. JS and Citadel are both top tier trading shops and they could not be more different when it come to wlb. It's not hard to sniff out during the process though.

@vineyardlabs: JS - Jane Street

Re: C++ patterns for low-latency applications including high-frequency trading

#214

Post by immibis »

idohft wrote: post #210
immibis wrote: post #171

Earlier quoted context omitted.

Yes, but you and I can't even talk to the exchange. We have to talk to one of many brokers that are allowed to talk to the exchange, and brokers do much more than just passing your orders to exchanges. For example, IIRC they can legally front-run your orders.

What exactly do you mean when you say this: > IIRC they can legally front-run your orders. I doubt this is true, but there are definitions attached to front-running.

What I was actually thinking about was the infamous Payment for Order Flow, where your broker doesn't actually send your orders to an exchange, but gets paid to send them to a giant HFT firm which can do whatever it wants with no oversight, including using the information that you are about to place an order to inform their algorithms which may result in them putting out a similar order before yours.

Re: C++ patterns for low-latency applications including high-frequency trading

#215

Post by rramadass »

neomantra wrote: post #185
rramadass wrote: post #110

Earlier quoted context omitted.

Any good books/resources you can recommend to learn about the above architectures/techniques?

Some years ago I wrote a gist about HFT/HPC systems patterns (versus OPs C++ patterns) applied to dockerized Redis. Might be dated, but touches on core isolation/pinning, numa/cgroups, kernel bypass, with some links to go deeper. Nowadays I do it with Kubernetes and Nomad facilities, but same basic ideas: https://gist.github.com/neomantra/3c9b89887d19be6fa5708bf401...

Nice; reminds me of the Redhat Performance Tuning and Real Time Low Latency Optimization guides.

Re: C++ patterns for low-latency applications including high-frequency trading

#216

Post by rramadass »

crabmusket wrote: post #187
rramadass wrote: post #110

Earlier quoted context omitted.

Any good books/resources you can recommend to learn about the above architectures/techniques?

A few episodes of Signals and Threads, a podcast from Jane Street, go into parts of it.

Thank You.

Re: C++ patterns for low-latency applications including high-frequency trading

#217

Post by deepsun »

deepsun wrote: post #35
fooker wrote: post #34

Earlier quoted context omitted.

You say it's easier in Rust, but you still have a complete C++ implementation and not a Rust one. :)

Linus said he wouldn't start Linux if Unix was ready at that time.

Don't know why so many downvotes, probably because I confused Unix with "GNU kernel".

Here's the source (Jan 29, 1992):

    If the GNU kernel had been ready last spring, I'd not have bothered to
    even start my project: the fact is that it wasn't and still isn't. Linux
    wins heavily on points of being available now.
https://groups.google.com/g/comp.os.minix/c/wlhw16QWltI/m/P8...

Re: C++ patterns for low-latency applications including high-frequency trading

#218

Post by nickelpro »

PoignardAzur wrote: post #195
nickelpro wrote: post #83

Fairly trivial base introduction to the subject. In my experience teaching undergrads they mostly get this stuff already. Their CompArch class has taught them the basics of branch prediction, cache coherence, and instruction caches; the trivial elements of performance. I'm somewhat surprised the piece doesn't deal at all with a classic performance killer, false sharing, although it seems mostly concerned with single-…

> optimization tricks like fat LTO, PGO, or even the standardized hinting attributes ([[likely]], [[unlikely]]) for optimizing icache layout If you do PGO, aren't hinting attributes counter-productive? In fact, the common wisdom I mostly see compiler people express is that most of the time they're counter-productive even without PGO, and modern compilers trust their own analysis passes more than they trust these hint…

I said "or even", if you're regularly using PGO they're irrelevant, but not everyone regularly uses PGO in a way that covers all their workloads.

The hinting attributes are exceptional for lone conditionals (not if/else trees) without obvious context to the compiler if it will frequently follow or skip the branch. Compilers are frequently conservative with such things and keep the code in the hot path.

The [[likely]] attribute then doesn't matter so much, but [[unlikely]] is absolutely respected and gets the code out of the hot path, especially with inlined into a large section. Godbolt is useful to verify this but obviously there's no substitute for benchmarking the performance impact.

Re: C++ patterns for low-latency applications including high-frequency trading

#219

Post by vineyardlabs »

ketanhwr wrote: post #213
kangda123 wrote: post #206

Earlier quoted context omitted.

Not OP but it varies wildly between companies. JS and Citadel are both top tier trading shops and they could not be more different when it come to wlb. It's not hard to sniff out during the process though.

@vineyardlabs: JS - Jane Street

Ah, duh

Re: C++ patterns for low-latency applications including high-frequency trading

#220

Post by jeffreyrogers »

akira2501 wrote: post #90
jeffreyrogers wrote: post #18

Earlier quoted context omitted.

Bid/ask spreads are far narrower than they were previously. If you look at the profits of the HFT industry as a whole they aren't that large (low billions) and their dollar volume is in the trillions. Hard to argue that the industry is wildly prosocial but making spreads narrower does mean less money goes to middlemen.

> but making spreads narrower does mean less money goes to middlemen. On individual trades. I would think you'd have to also argue that their high overall trading volume is somehow also a benefit to the broader market or at the very least that it does not outcompete the benefits of narrowing.

Someone is taking the other side of the trade. Presumably they have a reason for making that trade, so I don't see how higher volume makes people worse off. Probably some of those trades are wealth destroying (due to transaction costs) but it is destroying traders' and speculators' wealth, not some random person who can't afford it, since if you trade rarely your transaction costs are lower than before HFT became prominent.
Post reply on HN