Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C++ patterns for low-latency applications including high-frequency trading

arxiv.org

Post reply on HN
Print view View original

101–110 of 240 posts

Re: C++ patterns for low-latency applications including high-frequency trading

#101

Post by Maxatar »

nickelpro wrote: post #87
mbo wrote: post #84

Earlier quoted context omitted.

Out of interest, do you have any literature that you'd recommend instead?

On the software side I don't think HFT is as special a space as this paper makes it out to be.[1] Each year at cppcon there's another half-dozen talks going in depth on different elements of performance that cover more ground collectively than any single paper will. Similarly, there's an immense amount of formal literature and textbooks out of the game development space that can be very useful to newcomers looking fo…

As someone who does quant trading professionally and game development as a hobby, they both are performance sensitive, but they emphasize different kinds of performance. Trading is about minimizing latency while video games are about maximizing bandwidth.

Video games try to cram as much work as possible within about 16 milliseconds whereas for most trading algorithms 16 milliseconds is too slow to do anything, you want to process and produce a response to input within the span of microseconds, which is 3 orders of magnitude faster than a single frame in a video game.

The problem spaces really are quite distinct in this respect and a lot of techniques from one space don't really carry over to the other.

Re: C++ patterns for low-latency applications including high-frequency trading

#102

Post by _aavaa_ »

mbo wrote: post #84
nickelpro wrote: post #83

Fairly trivial base introduction to the subject. In my experience teaching undergrads they mostly get this stuff already. Their CompArch class has taught them the basics of branch prediction, cache coherence, and instruction caches; the trivial elements of performance. I'm somewhat surprised the piece doesn't deal at all with a classic performance killer, false sharing, although it seems mostly concerned with single-…

Out of interest, do you have any literature that you'd recommend instead?

I'd recommend: https://www.computerenhance.com

The author has a strong game development (engine and tooling) background and I have found it incredibly useful.

It also satisfies the requirement for "A genuine fear, hate, and anger, towards unnecessary allocations, copies, and other performance killers."

Re: C++ patterns for low-latency applications including high-frequency trading

#103

Post by usefulcat »

cheonic730 wrote: post #80
FredPret wrote: post #30

Earlier quoted context omitted.

What's the mechanism for removing liquidity?

> What's the mechanism for removing liquidity? Liquidity removal = market order Liquidity providing = limit order (not immediately executable)

The order type is a red herring (you can take liquidity with a limit order).

The only difference between an order that removes liquidity and an order that adds liquidity is whether it executes upon arrival (removing liquidity) or rests on the order book on arrival (adding liquidity).

Re: C++ patterns for low-latency applications including high-frequency trading

#104

Post by nickelpro »

Maxatar wrote: post #101
nickelpro wrote: post #87

Earlier quoted context omitted.

On the software side I don't think HFT is as special a space as this paper makes it out to be.[1] Each year at cppcon there's another half-dozen talks going in depth on different elements of performance that cover more ground collectively than any single paper will. Similarly, there's an immense amount of formal literature and textbooks out of the game development space that can be very useful to newcomers looking fo…

As someone who does quant trading professionally and game development as a hobby, they both are performance sensitive, but they emphasize different kinds of performance. Trading is about minimizing latency while video games are about maximizing bandwidth. Video games try to cram as much work as possible within about 16 milliseconds whereas for most trading algorithms 16 milliseconds is too slow to do anything, you wa…

Of course, time spans are different. But read the paper, it's dealing with incredibly basic techniques. When we're speaking on the level of "data should be in cache" and "cold code should be far away from the hot path" the fields are practically identical.

You're entirely correct that quant work is dealing with pure latency in situations where game engines might amortize throughput over multiple frames. But a question about "how do I get started in performance/latency work?" isn't usefully answered by "get a Hudson River Trading internship".

Re: C++ patterns for low-latency applications including high-frequency trading

#105

Post by Maxatar »

nickelpro wrote: post #104
Maxatar wrote: post #101

Earlier quoted context omitted.

As someone who does quant trading professionally and game development as a hobby, they both are performance sensitive, but they emphasize different kinds of performance. Trading is about minimizing latency while video games are about maximizing bandwidth. Video games try to cram as much work as possible within about 16 milliseconds whereas for most trading algorithms 16 milliseconds is too slow to do anything, you wa…

Of course, time spans are different. But read the paper, it's dealing with incredibly basic techniques. When we're speaking on the level of "data should be in cache" and "cold code should be far away from the hot path" the fields are practically identical. You're entirely correct that quant work is dealing with pure latency in situations where game engines might amortize throughput over multiple frames. But a questio…

On that point we certainly agree.

Re: C++ patterns for low-latency applications including high-frequency trading

#106

Post by CyberDildonics »

worstspotgain wrote: post #96
CyberDildonics wrote: post #89

Earlier quoted context omitted.

foo(const bar&) is ideal if you precisely wish to bar ownership. What? invariably it's more like when) you later decide to share ownership, shared_ptr shouldn't even be necessary for keeping track of single threaded scope based ownership. As for shared_ptrs being very rare, uh, no. We use them by the truckload. To each their own! You might want to look into that, you shouldn't need to count references in single threa…

> What? The context you're missing is in your post's GP. That poster holds a std::shared_ptr (for whatever perfectly valid reason) and wishes to pass it to foo(). However, he declares it as foo(const bar &) because the callee does not need to share in the ownership of the shared_ptr. That means it gets called as foo(*p.get()). > scope based ownership That's the incorrect assumption that you came to. Obviously if bar…

The context you're missing is in your post's GP.

I didn't miss any of that, that exactly what I thought it meant. I just don't know what you mean by precisely wish to bar ownership

That's the incorrect assumption that you came to.

Prove it. In a single threaded program with scope based ownership, that shared_ptr is going to be freed somewhere, so why not just have it exist in that scope as a unique_ptr so the ownership scope is clear?

Obviously if bar is only used for stack-based scope variables, no shared_ptr is needed.

Are you saying I'm wrong then saying the exact thing I just said?

Re: C++ patterns for low-latency applications including high-frequency trading

#107

Post by rramadass »

alchemist1e9 wrote: post #95
FredPret wrote: post #42

Earlier quoted context omitted.

If one outlawed / disincentivized hostile the bot behavior you described, there would still be the opportunity to do the good and profitable things I described.

These commenters have no clue what they are talking about. You don’t need to worry about hostile behaviors in the way they claim. These HN individuals are not domain educated and are spouting highly uninformed nonsense. Silly ideas like market orders take liquidity and limit orders provide it show an extremely rudimentary familiarity with only basic terminology of the field and with no understanding of the extremely…

> These HN individuals are not domain educated and are spouting highly uninformed nonsense.

Yeah, this is highly frustrating particularly for people like me who don't know anything about the domain i.e. HFT/Trading and would like to know more.

Can you recommend some good introductory books/resources ?

Re: C++ patterns for low-latency applications including high-frequency trading

#108

Post by Maxatar »

a_t48 wrote: post #70
Maxatar wrote: post #68

Earlier quoted context omitted.

> In function signatures, use const references: foo(const std::shared_ptr &p). This will prevent unnecessary bumps of the refcount. This advice doesn't seem quite right to me, and in my codebases I strictly forbid passing shared_ptr by const reference. If you don't need to share ownership of bar, then you do the following: foo(const bar&); If you do need to share ownership of bar, then you do the following: foo(std::…

If you _maybe_ need to share ownership, the second is a little pessimistic - you always increase the ref count.

That is correct and I can see that being a justification for passing a const&, in fact the C++ Core Guidelines agree with you that such a scenario is the only acceptable reason for passing a shared_ptr by const&, although they encourage passing by value, or just passing a const T&.

Re: C++ patterns for low-latency applications including high-frequency trading

#109

Post by hi_dang_ »

_aavaa_ wrote: post #102
mbo wrote: post #84

Earlier quoted context omitted.

Out of interest, do you have any literature that you'd recommend instead?

I'd recommend: https://www.computerenhance.com The author has a strong game development (engine and tooling) background and I have found it incredibly useful. It also satisfies the requirement for "A genuine fear, hate, and anger, towards unnecessary allocations, copies, and other performance killers."

Very anecdotal but of the people I know in game studios who are tasked with engine work, and people who make a killing doing FPGA work for HFT firms, both camps shook their head at Casey’s HMH thing. Uniformly I do not know of a single professional developer of this sort of caliber who looked at HMH and thought it looks great. Quite the opposite. I think they found his approach and justifications unsound as it would instil awful practices of premature unfounded optimization and a disdain for normal library code in favour of hand-rolling your own half-baked implementations based on outdated trivia. I agree with them on the basis that HMH exposed an unprepared and inexperienced audience to something that has to be regarded with utmost care. For this, I refer to Jonathan Blow’s presentation of “a list is probably good enough” as an antidote. I think JB’s recommendations are more in line with actual practices, whereas Casey just raised red flags uniformly from here-and-now engine devs shipping multi platform games.

Re: C++ patterns for low-latency applications including high-frequency trading

#110

Post by rramadass »

zxcvbn4038 wrote: post #62
worstspotgain wrote: post #50

Earlier quoted context omitted.

I briefly looked over your stock exchange code: - For memory management, consider switching to std::shared_ptr. It won't slow anything down and will put that concern to rest entirely. - For sockets, there are FOSS libraries that will outperform your code and save you a ton of headaches dealing with caveats and annoyances. For example, your looping through FD_ISSET is slower than e.g. epoll or kqueue. - For dependenci…

When I did low latency everyone was offloading TCP to dedicated hardware. They would shut down every single process on the server and bind the trading trading app to the CPUs during trading hours to ensure nothing interrupted. Electrons travel slower than light so they would rent server space at the exchange so they had direct access to the exchange network and didn't have to transverse miles of cables to send their…

Any good books/resources you can recommend to learn about the above architectures/techniques?
Post reply on HN