Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C++ patterns for low-latency applications including high-frequency trading

arxiv.org

Post reply on HN
Print view View original

231–240 of 240 posts

Re: C++ patterns for low-latency applications including high-frequency trading

#231

Post by resonious »

gohwell wrote: post #64
zxcvbn4038 wrote: post #62

Earlier quoted context omitted.

When I did low latency everyone was offloading TCP to dedicated hardware. They would shut down every single process on the server and bind the trading trading app to the CPUs during trading hours to ensure nothing interrupted. Electrons travel slower than light so they would rent server space at the exchange so they had direct access to the exchange network and didn't have to transverse miles of cables to send their…

I’ve worked at a few firms and never heard of an IT budget for f-ups. Sounds like a toxic work environment.

It's just business, no? Would you rather trade with a service that's liable for their mistakes or one that isn't?

Re: C++ patterns for low-latency applications including high-frequency trading

#233

Post by hermitdev »

vineyardlabs wrote: post #193
hermitdev wrote: post #188

Earlier quoted context omitted.

I can't go into too much detail (I currently work at an HFT, but not on the HFT bits), but some of the FPGAs make decisions without even seeing a entire packet coming in on the wire. Latencies are more often measured in nanos than micros from what I've seen lately.

Tangentially, how do you like working for an HFT? I'm a low level software / FPGA developer and have thought about going into the HFT space. Is the work life balance as terrible as people say?

It's really going to come down to which firm you're at and who your manager is. I'm at a smaller shop currently, and it's great. I basically work 8-5 most days, sometimes a little later if processes don't come up cleanly for market open at 5P (CME basically operates 23 hours a day with different trading sessions).

There are firms that are more like sweatshops. I spent roughly a decade at Citadel; when I started, average tenure was thought to be about 18 months. Hours were brutal, but it largely depended on your manager. I had one (he was fired 6 months after I started) that insisted on 80 hour weeks, ass in seat, no work from home, not even on the weekends. The kicker was, my team didn't even have the work at the time to justify all of us pulling 80 hour weeks. I also spent months at times pulling 100 hour weeks. That is, if you slept, it was at your desk (there were cots/sleeping bags available, but I never saw anyone bother). Maybe go home every 2nd or 3rd day for a change of clothes (you could shower at the office). Then, I had other managers where it was more 8-5. But, I was always on call 24/7, even while on vacation. That was the truly rough part. I would say on a whole, the reputation of working for Citadel is worse than it deserves, but it really matters who your manager is. That said, not sure I know anyone left at Citadel anymore other than Ken G. (I've been gone over a decade now).

My best advice when you're interviewing people, ask pointed questions about work-life balance. Avoid asking overly broad questions like "How's the work/life balance?". Instead prefer: "When was the last time you took a 2 or 3 week vacation?"

edit: At my current role, I do sometimes work outside of 8-5 and on weekends, but its because I choose to, because I'm working on something interesting that excites me. I'm not expected to, and no one gives me a hard time if I don't.

Re: C++ patterns for low-latency applications including high-frequency trading

#234

Post by dintech »

ssfrr wrote: post #178
dintech wrote: post #159

Earlier quoted context omitted.

Audio has a lot of buffering behaviour that you wouldn't generally see in event-reactive HFT. Think of all the plugins that you know of that have non-zero latency, compressors with 'lookahead' etc. There are maybe some similarities where the logic is more complex (loop unrolling, SIMD and so on) but I feel like plugins are generally optimizing for throughput (CPU usage) and quality (oversampling etc) rather than pure…

I guess the difference I'm interested in is whether HFT tends to be "realtime", in the sense that there's a hard deadline that you need to hit every time. Put another way, audio is operating on a much longer timescale, but cares a lot about worst-case latency (as well as throughput and quality). Is that true of HFT also, or are they more concerned with average latency? If computing a trade takes too long, can they ju…

Yes, there is a lot of effort to understand, for example, the 99th percentile tick-to-trade latency as well as the average latency. For a lot of those worst-case timings there are things that are occasionally out of your control like the kernel, the network, the hardware. Then there are things always in your control like how you implement your software.

Most HFT algos are busy-spinning. You'll see a core pinned at 100% as it checks for input availability over and over. The vast majority of the time it is actually doing nothing, from an external point of view. When that tick appears that it needs to react to, it springs into action. It might react to some input in the range of a few hundred nano seconds but for literally billions of nanoseconds it's doing nothing at all.

Audio is processing a buffer of data (usually whatever your audio interface buffer is set to), then waits patiently until the next buffer arrives. There's a regular pulse to it. If your CPU isn't hurting, there's way more time between those buffers than are required to process them. The order of magnitude between work and 'not work' for HFT is huge compared to audio generally. HFT logic is simple, small and fast. Audio is hopefully complex if you paid good money for it. The best way to go fast is to do almost nothing.

In terms of the impact of not meeting a particular deadline, it's an opportunity cost in HFT. In audio, you get audio dropouts if you can't process it fast enough. In HFT, if your reaction time is slow, you're not at the front of the queue for that juicy trading opportunity you and everyone else just spotted, then you miss it entirely and someone else gets the prize. HFT is all about making thousands of small statistically favourable bets and being able to execute on them fast enough to realise the opportunity before the next fastest guy.

Re: C++ patterns for low-latency applications including high-frequency trading

#235

Post by musicale »

TheAlchemist wrote: post #79
globular-toast wrote: post #5

Is there any good reason for high-frequency trading to exist? People often complain about bitcoin wasting energy, but oddly this gets a free pass despite this being a definite net negative to society as far as I can tell.

Because it's not explicitly forbidden ? I would argue that HFT is a rather small space, albeit pretty concentrated. It's several orders of magnitude smaller in terms of energy wasting than Bitcoin. The only positive from HFT is liquidity and tighter spreads, but it also depends what people put into HFT definition. For example, Robinhood and free trading, probably wouldn't exist without it. They are taking a part of t…

Buying and quickly selling a stock requires a buyer and a seller, so it doesn't seem like an intrinsic property that real liquidity would be increased. Moreover, fast automated trading seems likely to increase volatility.

I tend to agree that for long term investment it probably doesn't make a huge difference except for possible cumulative effects of decreased liquidity, increased volatility, flash crashes, etc. Also possibly a loss of small investor confidence since the game seems even more rigged in a way that they cannot compete with.

Re: C++ patterns for low-latency applications including high-frequency trading

#236

Post by musicale »

bravura wrote: post #10
globular-toast wrote: post #5

Is there any good reason for high-frequency trading to exist? People often complain about bitcoin wasting energy, but oddly this gets a free pass despite this being a definite net negative to society as far as I can tell.

Warren Buffett proposed that the stock market should be open less frequently, like once a quarter or similar. This would encourage long-term investing rather than reacting to speculation. Regardless, there are no natural events that necessitate high-frequency trading. The underlying value of things rarely changes very quickly, and if it does it's not volatile, rather it's a firm transiton.

The value of a stock usually doesn't change every millisecond. There doesn't seem to be a huge public need to pick winners and losers based on ping time.

Re: C++ patterns for low-latency applications including high-frequency trading

#237

Post by jqmp »

immibis wrote: post #214
idohft wrote: post #210

Earlier quoted context omitted.

What exactly do you mean when you say this: > IIRC they can legally front-run your orders. I doubt this is true, but there are definitions attached to front-running.

What I was actually thinking about was the infamous Payment for Order Flow, where your broker doesn't actually send your orders to an exchange, but gets paid to send them to a giant HFT firm which can do whatever it wants with no oversight, including using the information that you are about to place an order to inform their algorithms which may result in them putting out a similar order before yours.

FYI, this is false. These HFT firms are subject to the same regulation as other brokers, which is extensive. In particular, it's explicitly forbidden to front-run pending customer orders, or to share any information about them with the proprietary side of the firm. In my experience, these firms take these rules very seriously.

Re: C++ patterns for low-latency applications including high-frequency trading

#238

Post by jqmp »

ykonstant wrote: post #199

I am curious: why does this field use/used C++ instead of C for the logic? What benefits does C++ have over C in the domain? I am proficient in C/assembly but completely ignorant of the practices in HFT so please go easy on the explanations!

C++ is more expressive and allows much more abstraction than C. For a long time C++ was the only mainstream language that provided C-level performance as well as rich abstractions, which is why it became popular in fields that require complex domain modeling, like HFT, gamedev, and graphics. (Of course one can debate whether this expressivity is worth the enormous complexity of the language, but in practice people have empirically chosen C++.)
Post reply on HN