Live data from Hacker News

Low Latency Optimization: Understanding Pages (Part 1)

hudsonrivertrading.com

41–50 of 59 posts

Re: Low Latency Optimization: Understanding Pages (Part 1)

#41

Earlier quoted context omitted.

Your comment is correct but might cause readers to underestimate how annoying this tuning work is and how difficult it is to get everything into hugepages (executable memory and stack memory and shared libraries if applicable, not just specific heap allocations). We are trading a joke asset class on joke venues that have millisecond-scale jitter, so we can get away with using io_uring instead of kernel bypass network…

Just wondering, how useful is it to get code and stack memory into hugepages? I thought you usually access them sequentially so it doesn't matter that much to put them in hugepages.

Wrt code, look at the bench in the article. Even with sequential access, you can get a decent speedup using huge pages. But unless you have a good profile and using PGO, it'll likely not be that sequential for code. Like everything else you'll need to measure it to know exactly what benefit you might get. As a starting point, you can start at looking at the itlb load misses with perf stat -d

Stack access is another story as it's usually local and sequential so it might not be that useful.

Re: Low Latency Optimization: Understanding Pages (Part 1)

#42
If any of this is interesting to you, but you would like some deeper content to bite into, perhaps start here:

https://lmax-exchange.github.io/disruptor/disruptor.html

This technical paper sent me on a multi-year journey regarding one simple question: "If this stuff is fast enough for fintech, why can't we make everything work this way?" Handling millions of requests per second on 1 thread is well beyond the required performance envelope for most public "webscale" products.

Re: Low Latency Optimization: Understanding Pages (Part 1)

#43

So many puzzling things here, from the brand new user account created to post this (a portmanteau of Jump Trading and Citadel), to the very minimal information presented (even my own article on it for my software covers about as much), to the people in the comments here conflating virtual memory with hard disk paging in spite of TFA, to red herring comments about RT scheduling, ...

Performance is a shiny toy that attracts and distracts many from understanding the fundamentals. Interestingly, understanding fundamentals is a pre-requisite to understanding performance!

Re: Low Latency Optimization: Understanding Pages (Part 1)

#44

So many puzzling things here, from the brand new user account created to post this (a portmanteau of Jump Trading and Citadel), to the very minimal information presented (even my own article on it for my software covers about as much), to the people in the comments here conflating virtual memory with hard disk paging in spite of TFA, to red herring comments about RT scheduling, ...

URL for your article?

https://www.chaoticafractals.com/manual/getting-started/enab...

Admittedly it's a bit terse, but at least it gives some steps you can use to enable it on Windows. It also benefits other software, such as 7zip. I need to update the page because these days the performance benefits are larger, due to the ever-widening divide between compute and memory speeds, CPUs having bigger large page TLBs, and additional optimisations...

Re: Low Latency Optimization: Understanding Pages (Part 1)

#45

I've been working for HFT firms since I moved to NYC over a decade ago. The article looks like a good summation HugePage benefits (I'm a sysadmin, not a programmer so I understand it on a topical level only). What I do find fascinating is that HRT is actively blogging about this stuff. Ten years ago, everyone in the biz was super secretive and never made any public announcement about what we did - even stuff that I w…

> everyone in the biz was super secretive [..] even stuff that I would take from [..] (which were public knowledge)

It's not just Finance but other industries have this culture as well. I suspect it manifests in an environment that is perceived to be hyper competitive--any perceived advantage regardless of where it came from or how differentiated it is, is held closely and over-weighed if proper metrics aren't in place to continue pushing for improvement.

Re: Low Latency Optimization: Understanding Pages (Part 1)

#46

Earlier quoted context omitted.

URL for your article?

https://www.chaoticafractals.com/manual/getting-started/enab... Admittedly it's a bit terse, but at least it gives some steps you can use to enable it on Windows. It also benefits other software, such as 7zip. I need to update the page because these days the performance benefits are larger, due to the ever-widening divide between compute and memory speeds, CPUs having bigger large page TLBs, and additional optimisati…

I am biased but I don't think it's fair to say that your article covers as much. There is more content in the article written in a way that's trying to be approachable. I certainly will agree it's wordy but it's hard to make content that's interesting to a wide audience

Your article does cover how to use/enable them while the post does not. But it's meant for part 2

Re: Low Latency Optimization: Understanding Pages (Part 1)

#47

Earlier quoted context omitted.

https://www.chaoticafractals.com/manual/getting-started/enab... Admittedly it's a bit terse, but at least it gives some steps you can use to enable it on Windows. It also benefits other software, such as 7zip. I need to update the page because these days the performance benefits are larger, due to the ever-widening divide between compute and memory speeds, CPUs having bigger large page TLBs, and additional optimisati…

I am biased but I don't think it's fair to say that your article covers as much. There is more content in the article written in a way that's trying to be approachable. I certainly will agree it's wordy but it's hard to make content that's interesting to a wide audience Your article does cover how to use/enable them while the post does not. But it's meant for part 2

Agreed, "about as much" is a bit unfair.

Re: Low Latency Optimization: Understanding Pages (Part 1)

#48

So many puzzling things here, from the brand new user account created to post this (a portmanteau of Jump Trading and Citadel), to the very minimal information presented (even my own article on it for my software covers about as much), to the people in the comments here conflating virtual memory with hard disk paging in spite of TFA, to red herring comments about RT scheduling, ...

Performance is a shiny toy that attracts and distracts many from understanding the fundamentals. Interestingly, understanding fundamentals is a pre-requisite to understanding performance!

I'd argue that understanding what happens on every single memory access qualifies as fundamental.

Re: Low Latency Optimization: Understanding Pages (Part 1)

#49

Earlier quoted context omitted.

It can be done by manually remapping the relevant sections upon application startup. Perhaps [1] is a good resource to start with (page nr. 7). Example code is here [2]. And [3] makes some experiments with it. [1] https://www.kernel.org/doc/ols/2006/ols2006v2-pages-83-90.pd... [2] https://github.com/intel/iodlr/blob/master/large_page-c/exam... [3] https://easyperf.net/blog/2022/09/01/Utilizing-Huge-Pages-Fo...

Thanks, cool stuff. Especially liblppreload.so described in [2] and [3]. I'll give it a try. Do you have any tips how to achieve the same for the stack?

IMO the easiest way (but certainly not the only way) is to allocate a new stack and switch to it with makecontext(). The manpage has a full code example. You just need to change the stack alloc. This approach has a few drawbacks but is hard to beat in terms of simplicity.

Re: Low Latency Optimization: Understanding Pages (Part 1)

#50

This article is pretty thin but it's not wrong. If you're interested in consistent low latency you do need to avoid TLB misses, and also page faults, cache contention, cache coherency delay (making sure no other cores are accessing your memory) from the CC protocol (MOESI/MESI(F)) and mis-prediction, and that's after you have put all your core's threads into SCHED_FIFO. Using https://lttng.org/ can be really helpful…

Again, I am biased. But the article explains mem translation in fairly simple terms, hammers the main advantages of HPs (better use of the TLB, simpler and smaller PT). Explains clearly what how much mem the TLB can cover, what a page walk is and how much time it takes (before even loading actual data), the importance of the cache wrt PT. It shows some perf numbers of random vs iterative mem accesses.

I don't think you'll find many articles that detail these points. Now, they might be trivial to you and that's totally fair. But the goal is to address a wide audience. Additionally, the article is not addressing how to use HPs but that's for part 2.

Wrt to other points, I certainly agree they are important topics to explore. I would add using perf is super important to easily access the perf counters

Post reply on HN