Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C++ patterns for low-latency applications including high-frequency trading

arxiv.org

Post reply on HN
Print view View original

41–50 of 240 posts

Re: C++ patterns for low-latency applications including high-frequency trading

#41

Post by munk-a »

pi-rat wrote: post #23
globular-toast wrote: post #5

Is there any good reason for high-frequency trading to exist? People often complain about bitcoin wasting energy, but oddly this gets a free pass despite this being a definite net negative to society as far as I can tell.

Generally gets attributed with: - Increased liquidity. Ensures there's actually something to be traded available globally, and swiftly moves it to places where it's lacking. - Tighter spreads, the difference between you buying and then selling again is lower. Which often is good for the "actual users" of the market. - Global prices / less geographical differences in prices. Generally you can trust you get the right p…

> Tighter spreads, the difference between you buying and then selling again is lower. Which often is good for the "actual users" of the market.

I just wanted to highlight this one in particular - the spread is tighter because HFTs eat the spread and reduce the error that market players can benefit from. The spread is disappearing because of rent-seeking from the HFTs.

Re: C++ patterns for low-latency applications including high-frequency trading

#42

Post by FredPret »

jeffbee wrote: post #17
FredPret wrote: post #7

Earlier quoted context omitted.

Non-bitcoin transactions are just a couple of entries in various databases. Mining bitcoin is intense number crunching. HFT makes the financial markets a tiny bit more accurate by resolving inconsistencies (for example three pairs of currencies can get out of whack with one another) and obvious mispricings (for various definitions of "obvious")

That's a nice fairy tale that they probably tell their kids when asked, but what the profitable firms are doing at the cutting edge is inducing responses in the other guys' robots, in a phase that the antagonist controls, then trading against what they know is about to happen. It is literally market manipulation. A way to kill off this entire field of endeavor is to charge a tax on cancelled orders.

If one outlawed / disincentivized hostile the bot behavior you described, there would still be the opportunity to do the good and profitable things I described.

Re: C++ patterns for low-latency applications including high-frequency trading

#43

Post by nine_k »

sneilan1 wrote: post #40
temporarely wrote: post #36

Earlier quoted context omitted.

fun fact: the original LMAX was designed for and written in Java. https://martinfowler.com/articles/lmax.html

I think it made sense at the time. From what I understand, you can make Java run as fast as C++ if you're careful with it and use JIT. However, I have never tried such a thing and my source is hearsay from friends who have worked in financial institutions. Then you get added benefit of the Java ecosystem.

From my hearsay, you absolutely can, given two things: fewer pointer-chasing data structures, and, most crucially, fewer or no allocations. Pre-allocate arrays of things you need, run ring buffers on them if you have to use a varying number of things.

A fun but practical approach which I again heard (second-hand) to be used, is just drowning your code in physical RAM, and switch the GC completely off. Have enough RAM to run a trading day, then reboot. The cost is trivial, compared to spending engineering hours on different approaches.

Re: C++ patterns for low-latency applications including high-frequency trading

#44

Post by nine_k »

munificent wrote: post #16
jeffreygoesto wrote: post #2

Reminds me of https://github.com/CppCon/CppCon2017/blob/master/Presentatio...

This is an excellent slideshow. The slide on measuring by having a fake server replaying order data, a second server calculating runtimes, the server under test, and a hardware switch to let you measure packet times is so delightfully hardcore. I don't have any interest in working in finance, but it must be fun working on something so performance critical that buying a rack of hardware just for benchmarking is econom…

Delightfully hardcore indeed!

But of course you don't have to buy a rack of servers for testing, you can rent it. Servers are a quickly depreciating asset, why invest in them?

Re: C++ patterns for low-latency applications including high-frequency trading

#45

Post by a_t48 »

munificent wrote: post #16
jeffreygoesto wrote: post #2

Reminds me of https://github.com/CppCon/CppCon2017/blob/master/Presentatio...

This is an excellent slideshow. The slide on measuring by having a fake server replaying order data, a second server calculating runtimes, the server under test, and a hardware switch to let you measure packet times is so delightfully hardcore. I don't have any interest in working in finance, but it must be fun working on something so performance critical that buying a rack of hardware just for benchmarking is econom…

The self driving space does this :)

Re: C++ patterns for low-latency applications including high-frequency trading

#46

Post by a_t48 »

nine_k wrote: post #44
munificent wrote: post #16

Earlier quoted context omitted.

This is an excellent slideshow. The slide on measuring by having a fake server replaying order data, a second server calculating runtimes, the server under test, and a hardware switch to let you measure packet times is so delightfully hardcore. I don't have any interest in working in finance, but it must be fun working on something so performance critical that buying a rack of hardware just for benchmarking is econom…

Delightfully hardcore indeed! But of course you don't have to buy a rack of servers for testing, you can rent it. Servers are a quickly depreciating asset, why invest in them?

You'd want it to be the exact same hardware as in production, for one.

Re: C++ patterns for low-latency applications including high-frequency trading

#47

Post by rcxdude »

globular-toast wrote: post #5

Is there any good reason for high-frequency trading to exist? People often complain about bitcoin wasting energy, but oddly this gets a free pass despite this being a definite net negative to society as far as I can tell.

Yes: it put a much larger, more expensive, and less efficient part of wall street out of business. Before it was done with computers, it was done with lots of people doing the same job. What was that job? Well, if you want to go to a market and sell something, you generally would like for there to be someone there who's buying it. But it's not always the case that there's a buyer there right at the time who actually wants the item for their own use. The inverse is also true for a prospective buyer. Enter middle-men or market makers who just hang around the marketplace, learning roughly how much people will buy or sell a given good for, and buy it for slightly less than they can sell it later for. This is actually generally useful if you just want to buy or sell something.

Now, does this need to get towards milli-seconds or nano-seconds? No, this is just the equivalent of many of these middle-men racing to give you an offer. But it's (part of) how they compete with each other, and as they do so they squeeze the margins of the industry as a whole: In fact the profits of HFT firms have decreased as a percentage of the overall market and in absolute terms after the initial peak as they displaced the day traders doing the same thing.

Re: C++ patterns for low-latency applications including high-frequency trading

#48

Post by hi_dang_ »

sneilan1 wrote: post #32
JedMartin wrote: post #27

Earlier quoted context omitted.

It's not easy to get data structures like this right in C++. There are a couple of problems with your implementation of the queue. Memory accesses can be reordered by both the compiler and the CPU, so you should use std::atomic for your producer and consumer positions to get the barriers described in the original LMAX Disruptor paper. In the get method, you're returning a pointer to the element within the queue after…

>> In the get method, you're returning a pointer to the element within the queue after bumping the consumer position (which frees the slot for the producer), so it can get overwritten while the user is accessing it. And then your producer and consumer positions will most likely end up in the same cache line, leading to false sharing. I did not realize this. Thank you so much for pointing this out. I'm going to take a…

I sincerely doubt the big HFT firms use anything of Fowler’s. Their optimizations are down to making their own hardware. LL is very context dependent and Amdahl’s law applies here.

Re: C++ patterns for low-latency applications including high-frequency trading

#49

Post by fooker »

sneilan1 wrote: post #37
fooker wrote: post #34

Earlier quoted context omitted.

You say it's easier in Rust, but you still have a complete C++ implementation and not a Rust one. :)

It took me about a year to build all of this stuff in C++. So I imagine since I've had to learn rust, it will probably take me the same amount of time if I can save time with dependencies.

In my experience, once you know the problem really well, yes you're right.

If you are building a complex prototype from scratch, you'll usually spend more time fighting the Rust compiler than trying out alternate design decisions.

Re: C++ patterns for low-latency applications including high-frequency trading

#50

Post by worstspotgain »

sneilan1 wrote: post #3

I've got an implementation of a stock exchange that uses the LMAX disruptor pattern in C++ https://github.com/sneilan/stock-exchange And a basic implementation of the LMAX disruptor as a couple C++ files https://github.com/sneilan/lmax-disruptor-tutorial I've been looking to rebuild this in rust however. I reached the point where I implemented my own websocket protocol, authentication system, SSL etc. Then I realized…

I briefly looked over your stock exchange code:

- For memory management, consider switching to std::shared_ptr. It won't slow anything down and will put that concern to rest entirely.

- For sockets, there are FOSS libraries that will outperform your code and save you a ton of headaches dealing with caveats and annoyances. For example, your looping through FD_ISSET is slower than e.g. epoll or kqueue.

- For dependencies, C++ is definitely wilder than other languages. Dependencies are even harder to find than they are to manage. There's a lot of prospective library code, some of it hidden in little forgotten folds of the Internet. Finding it is basically a skill unto itself, one that can pay off handsomely.

Post reply on HN