Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Low Latency Optimization: Understanding Pages (Part 1)

hudsonrivertrading.com

Post reply on HN
Print view View original

21–30 of 59 posts

Re: Low Latency Optimization: Understanding Pages (Part 1)

#21

Post by vgatherps »

aldanor wrote: post #18
vgatherps wrote: post #9

Earlier quoted context omitted.

Stock / near stock Linux is pretty close to fine for HFT. You basically only interact with the kernel on init/shutdown or outside of the fast path, and do something like isolcpus to delegate the kernel and interrupt handling to some garbage cores and give you the rest to do what you want with.

Not really. Most HFTs would choose some sort of kernel bypass for critical path networking needs. (unless that's what you mean by near stock)

Nothing needs to be changed about your kernel to bypass it.

You can outright install the openonload drivers, preload to intercept epoll, and it literally just works.

Going to efvi can cut out ~1us but that requires specifically targeting efvi, and has more operational / code setup pain. Works on stock Linux all the same though

Re: Low Latency Optimization: Understanding Pages (Part 1)

#22

Post by pixelpoet »

So many puzzling things here, from the brand new user account created to post this (a portmanteau of Jump Trading and Citadel), to the very minimal information presented (even my own article on it for my software covers about as much), to the people in the comments here conflating virtual memory with hard disk paging in spite of TFA, to red herring comments about RT scheduling, ...

Re: Low Latency Optimization: Understanding Pages (Part 1)

#23

Post by JedMartin »

menaerus wrote: post #20
JedMartin wrote: post #16

Earlier quoted context omitted.

The part about getting everything into hugepages sounds interesting. Any idea where can I find some resources on that? Most of what I was able to find only tell you how to do that for heap allocations.

It can be done by manually remapping the relevant sections upon application startup. Perhaps [1] is a good resource to start with (page nr. 7). Example code is here [2]. And [3] makes some experiments with it. [1] https://www.kernel.org/doc/ols/2006/ols2006v2-pages-83-90.pd... [2] https://github.com/intel/iodlr/blob/master/large_page-c/exam... [3] https://easyperf.net/blog/2022/09/01/Utilizing-Huge-Pages-Fo...

Thanks, cool stuff. Especially liblppreload.so described in [2] and [3]. I'll give it a try. Do you have any tips how to achieve the same for the stack?

Re: Low Latency Optimization: Understanding Pages (Part 1)

#24

Post by pixelpoet »

up2isomorphism wrote: post #6

So useless.

I'm with you: zero lines of code presented for the most verbose description on the topic I've seen. No mention of other software that benefits from it, no actual latency graphs (it's somewhat implied by the throughput graph), only one CPU measured, ...

Re: Low Latency Optimization: Understanding Pages (Part 1)

#25

Post by nnoitra »

angry_octet wrote: post #12

This article is pretty thin but it's not wrong. If you're interested in consistent low latency you do need to avoid TLB misses, and also page faults, cache contention, cache coherency delay (making sure no other cores are accessing your memory) from the CC protocol (MOESI/MESI(F)) and mis-prediction, and that's after you have put all your core's threads into SCHED_FIFO. Using https://lttng.org/ can be really helpful…

No post body was provided.

Re: Low Latency Optimization: Understanding Pages (Part 1)

#26

Post by WhiteBlueSkies »

pixelpoet wrote: post #24
up2isomorphism wrote: post #6

So useless.

I'm with you: zero lines of code presented for the most verbose description on the topic I've seen. No mention of other software that benefits from it, no actual latency graphs (it's somewhat implied by the throughput graph), only one CPU measured, ...

Lol, this is the very definition of interesting. They included source code here: https://github.com/hudson-trading/hrtbeat/blob/master/huge_m....

Re: Low Latency Optimization: Understanding Pages (Part 1)

#27

Post by WhiteBlueSkies »

zozbot234 wrote: post #7

If you're doing truly low latency stuff you shouldn't be swapping at all, everything should be 100% resident in memory at all times. So "pages" are totally irrelevant to you. (You should also probably be using something like the PREEMPT_RT patchset, adjust scheduling priorities and try your best to ensure that the CPU core(s) your app is running on aren't burdened by serving interrupts. Plus likely a lot of other stu…

I'm not entirely sure you understand memory hierarchy and that RAM is volatile so paging has to happen and keep in mind that reading memory from disk is a LOT slower than from main memory.

Re: Low Latency Optimization: Understanding Pages (Part 1)

#28

Post by pca006132 »

gavinray wrote: post #5

An OS page size is such a prevalent notion in software it's shocking I was oblivious to this a year ago before I got interested in database internals Something that I found interesting, there's a recent presentation by Neumann about the Umbra DBMS where he fields a question about hugepages at the end. I recall him saying they don't use it, which I found interesting. I know Oracle and MySQL recommended Transparent Hug…

You need to understand the bottleneck to determine whether or not huge pages are useful. THP requires the kernel to do additional work to give you the huge pages, which is usually more expensive then allocating huge pages through mmap directly.

Re: Low Latency Optimization: Understanding Pages (Part 1)

#29

Post by pca006132 »

anonymoushn wrote: post #10
vgatherps wrote: post #9

Earlier quoted context omitted.

Stock / near stock Linux is pretty close to fine for HFT. You basically only interact with the kernel on init/shutdown or outside of the fast path, and do something like isolcpus to delegate the kernel and interrupt handling to some garbage cores and give you the rest to do what you want with.

Your comment is correct but might cause readers to underestimate how annoying this tuning work is and how difficult it is to get everything into hugepages (executable memory and stack memory and shared libraries if applicable, not just specific heap allocations). We are trading a joke asset class on joke venues that have millisecond-scale jitter, so we can get away with using io_uring instead of kernel bypass network…

Just wondering, how useful is it to get code and stack memory into hugepages? I thought you usually access them sequentially so it doesn't matter that much to put them in hugepages.

Re: Low Latency Optimization: Understanding Pages (Part 1)

#30

Post by Bluecobra »

SkipperCat wrote: post #19

I've been working for HFT firms since I moved to NYC over a decade ago. The article looks like a good summation HugePage benefits (I'm a sysadmin, not a programmer so I understand it on a topical level only). What I do find fascinating is that HRT is actively blogging about this stuff. Ten years ago, everyone in the biz was super secretive and never made any public announcement about what we did - even stuff that I w…

> Now, I'm seeing HFT companies post articles like this and I'm thinking it has to be for recruiting. Why else would they do it?

I think you’re right in the money here. All the secret sauce is in FPGA trading now so there’s nothing secret about sharing this info.

Post reply on HN