Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C++ patterns for low-latency applications including high-frequency trading

arxiv.org

Post reply on HN
Print view View original

121–130 of 240 posts

Re: C++ patterns for low-latency applications including high-frequency trading

#121

Post by CyberDildonics »

a_t48 wrote: post #119
CyberDildonics wrote: post #106

Earlier quoted context omitted.

The context you're missing is in your post's GP. I didn't miss any of that, that exactly what I thought it meant. I just don't know what you mean by precisely wish to bar ownership That's the incorrect assumption that you came to. Prove it. In a single threaded program with scope based ownership, that shared_ptr is going to be freed somewhere, so why not just have it exist in that scope as a unique_ptr so the ownersh…

Not sure if this is what GP was getting at, but in games a shared pointer (or similar custom resource handle) to something like a texture can be very useful - you want to share resources between objects in the scene so that you don't load a thing from disk when it's already in memory, but don't want to hold onto it forever to save RAM.

Very true, and in between threads and in other areas too, but this isn't scope based ownership, it's completely separate from the scope hierarchy of the program.

Scope based ownership would be a unique_ptr that frees the heap memory when it goes out of scope (if it isn't moved).

Re: C++ patterns for low-latency applications including high-frequency trading

#122

Post by contingencies »

nickelpro wrote: post #83

Fairly trivial base introduction to the subject. In my experience teaching undergrads they mostly get this stuff already. Their CompArch class has taught them the basics of branch prediction, cache coherence, and instruction caches; the trivial elements of performance. I'm somewhat surprised the piece doesn't deal at all with a classic performance killer, false sharing, although it seems mostly concerned with single-…

How I might approach it. Interested in feedback from people closer to the space.

First, split the load in to simple asset-specific data streams with a front-end FPGA for raw speed. Resist the temptation to actually execute here as the friction is too high for iteration, people, supply chain, etc. Input may be a FIX stream or similar, output is a series of asset-specific binary event streams along low-latency buses, split in to asset-specific segments of a scalable cluster of low-end MCUs. Second, get rid of the general purpose operating system assumption on your asset-specific MCU-based execution platform to enable faster turnaround using low-level code you can actually find people to write on hardware you can actually purchase. Third, profit? In such a setup you'd need to monitor the overall state with a general purpose OS based governor which could pause or change strategies by reprogramming the individual elements as required.

Just how low are the latencies involved? At a certain point you're better off paying to get the hardware closer to the core than bothering with engineering, right? I guess that heavily depends on the rules and available DCs / link infrastructure offered by the exchanges or pools in question. I would guess a number of profitable operations probably don't disclose which pools they connect to and make a business of front-running, regulations or terms of service be damned. In such cases, the relative network geographic latency between two points of execution is more powerful than the absolute latency to one.

Re: C++ patterns for low-latency applications including high-frequency trading

#123

Post by FredPret »

alchemist1e9 wrote: post #92
jeffbee wrote: post #17

Earlier quoted context omitted.

That's a nice fairy tale that they probably tell their kids when asked, but what the profitable firms are doing at the cutting edge is inducing responses in the other guys' robots, in a phase that the antagonist controls, then trading against what they know is about to happen. It is literally market manipulation. A way to kill off this entire field of endeavor is to charge a tax on cancelled orders.

Exchanges put limits on cancellation rates as measured by a multiple of filled orders. You have to allow strategies that can induce other strategies as by definition those also increase liquidity. It’s a difficult problem to explain to anyone except the very few people who can understand the extremely complicated feedback loops that result from bots fighting bots, however the regulators actually have access to counte…

The other fundamental thing is that if you can’t sell something, you don’t really own it.

So wether pairs of people want to buy-and-hold or HFT their assets is really neither here nor there for uninvolved third parties.

Re: C++ patterns for low-latency applications including high-frequency trading

#124

Post by CyberDildonics »

worstspotgain wrote: post #117
CyberDildonics wrote: post #106

Earlier quoted context omitted.

The context you're missing is in your post's GP. I didn't miss any of that, that exactly what I thought it meant. I just don't know what you mean by precisely wish to bar ownership That's the incorrect assumption that you came to. Prove it. In a single threaded program with scope based ownership, that shared_ptr is going to be freed somewhere, so why not just have it exist in that scope as a unique_ptr so the ownersh…

> I just don't know what you mean by precisely wish to bar ownership If foo() doesn't need to share ownership now but may need to later, declaring it as foo(const std::shared_ptr &) instead of foo(const bar &) allows this change without revising any prototypes. However, if we precisely wish to prohibit shared ownership by foo(), we can do so by declaring it as foo(const bar &). > Prove it. In a single threaded progra…

If foo() doesn't need to share ownership now but may need to later,

This doesn't make sense. Why would a function in a more narrow scope need to take or share ownership? The variable passed will persist before and after the function call.

The incorrect assumption that you came to is that we were talking about stack variables.

I don't know what this means here. I said scope, you are saying stack. If lifetime is being dictated purely by scope, you don't need shared_ptr, you can just use a unique_ptr at the correct scope.

But anyways, here's an example that's both scope-based and single threaded: std::vector > v;

This isn't scope based lifetime, this is lifetime based on some other criteria than going out of scope. I will show you with what you wrote:

and another that duplicates and removes elements according to some ongoing criteria.

Re: C++ patterns for low-latency applications including high-frequency trading

#125

Post by steve1977 »

ssfrr wrote: post #114
Maxatar wrote: post #101

Earlier quoted context omitted.

As someone who does quant trading professionally and game development as a hobby, they both are performance sensitive, but they emphasize different kinds of performance. Trading is about minimizing latency while video games are about maximizing bandwidth. Video games try to cram as much work as possible within about 16 milliseconds whereas for most trading algorithms 16 milliseconds is too slow to do anything, you wa…

I'm curious how HFT relates to pro audio programming. The timescale is close to gaming (usually It's not hard real-time like you're going to crash your car, but if you miss your deadline it causes an unacceptable and audible glitch. I've always been a bit surprised that Jane Street uses OCaml. I know they've put a lot of attention into the GC, but it still seems fundamentally indeterminate in a way that would make mo…

In audio, you can usually tolerate a few milliseconds of latency (more when mixing, less when recording). This means you can buffer your audio stream, for example in blocks of 64 or 128 samples.

As far as I know, these millisecond latencies are orders of magnitude higher than what would be tolerable in HFT. There the units are microseconds and nanoseconds.

Re: C++ patterns for low-latency applications including high-frequency trading

#126

Post by shaklee3 »

zxcvbn4038 wrote: post #62
worstspotgain wrote: post #50

Earlier quoted context omitted.

I briefly looked over your stock exchange code: - For memory management, consider switching to std::shared_ptr. It won't slow anything down and will put that concern to rest entirely. - For sockets, there are FOSS libraries that will outperform your code and save you a ton of headaches dealing with caveats and annoyances. For example, your looping through FD_ISSET is slower than e.g. epoll or kqueue. - For dependenci…

When I did low latency everyone was offloading TCP to dedicated hardware. They would shut down every single process on the server and bind the trading trading app to the CPUs during trading hours to ensure nothing interrupted. Electrons travel slower than light so they would rent server space at the exchange so they had direct access to the exchange network and didn't have to transverse miles of cables to send their…

You don't need to shut down processes on the server. All you have to do is isolate CPU cores and move your workloads onto those cores. That's been a common practice in low latency networking for decades.

Re: C++ patterns for low-latency applications including high-frequency trading

#127

Post by shaklee3 »

sneilan1 wrote: post #59
worstspotgain wrote: post #50

Earlier quoted context omitted.

I briefly looked over your stock exchange code: - For memory management, consider switching to std::shared_ptr. It won't slow anything down and will put that concern to rest entirely. - For sockets, there are FOSS libraries that will outperform your code and save you a ton of headaches dealing with caveats and annoyances. For example, your looping through FD_ISSET is slower than e.g. epoll or kqueue. - For dependenci…

I did not know std::shared_ptr would not slow things down. I've learned something new today! :) Yes, I agree, epoll is a lot better than FD_ISSET. Maybe I can keep moving with my C++ code but do people still trust C++ projects anymore? My ideal use case is a hobbyist who wants a toy stock exchange to run directly in AWS. I felt that C++ has a lot of bad publicity and if I want anyone to trust/try my code I would have…

That's not true. It does slow things down because it has an atomic access. How slow depends on the platform.

unique_ptr does not slow things down.

Re: C++ patterns for low-latency applications including high-frequency trading

#128

Post by mrbald »

torlok wrote: post #8
globular-toast wrote: post #5

Is there any good reason for high-frequency trading to exist? People often complain about bitcoin wasting energy, but oddly this gets a free pass despite this being a definite net negative to society as far as I can tell.

To go a step further, I don't think you should be required to talk to middlemen when buying stocks, yet here we are. The house wants its cut.

Central counterparty concept implemented by most exchanges is a valid service, as otherwise counterparty risk management would be a nightmare - an example of a useful “middleman”.

Re: C++ patterns for low-latency applications including high-frequency trading

#129

Post by foobazgt »

FredPret wrote: post #57
arcimpulse wrote: post #6

Earlier quoted context omitted.

It would be trivial (and vastly more equitable) to quantize trade times.

You mean like settling trades every 0.25 seconds or something like that? Wouldn't there be a queue of trades piling up every 0.25 seconds, incentivizing maximum speed anyway?

Usually the proposal is to randomize the processing of the queue. So, as long as your trades get in during the window, there's no advantage to getting in any earlier. In theory the window is so small as to not have any impact on liquidity but wide enough to basically shut down all HFT.

Re: C++ patterns for low-latency applications including high-frequency trading

#130

Post by josefx »

bb88 wrote: post #97
throwaway2037 wrote: post #82

Earlier quoted context omitted.

In my experience: Allocation is OK, but garbage collection is bad.

I think back then GC defaulted running potentially at allocation. shared_ptr is a much better solution for garbage collection. One I wish that java had implemented.

I think CPython does reference counting for its memory management, it still has to run a GC since reference counting does not handle reference cycles.
Post reply on HN