Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta’s renewed commitment to jemalloc

engineering.fb.com

Post reply on HN
Print view View original

221–230 of 259 posts

Re: Meta’s renewed commitment to jemalloc

#221

Post by adsharma »

genxy wrote: post #178
adsharma wrote: post #72

> We plan to deliver improvements to [..] purging mechanisms During my time at Facebook, I maintained a bunch of kernel patches to improve jemalloc purging mechanisms. It wasn't popular in the kernel or the security community, but it was more efficient on benchmarks for sure. Many programs run multiple threads, allocate in one and free in the other. Jemalloc's primary mechanism used to be: madvise the page back to th…

What metrics were improved by your patches?

Some more historical context. It wasn't a random optimization idea that I thought about in the shower and implemented the next day. Previous work on company wide profiling, where my contribution was low level perf_events plumbing:

https://research.google/pubs/google-wide-profiling-a-continu... https://engineering.fb.com/2025/01/21/production-engineering...

The profiling clearly showed kernel functions doing memzero at the top of the profiles which motivated the change. The performance impact (A/B testing and measuring the throughput) also showed a benefit at the point the change was committed.

This was when "facebook" was a ~1GB ELF binary. https://en.wikipedia.org/wiki/HipHop_for_PHP

The change stopped being impactful sometime after 2013, when a JIT replaced the transpiler. I'm guessing likely before 2016 when continuous deployment came into play. But that was continuously deploying PHP code, not HHVM itself.

By the time the patches were reevaluated I was working on a Graph Database, which sounded a lot more interesting than going back to my old job function and defending a patch that may or may not be relevant.

I'm still working on one. Guilty as charged of carrying ideas in my head for 10+ years and acting on them later. Link in my profile.

Re: Meta’s renewed commitment to jemalloc

#222

Post by jcalvinowens »

menaerus wrote: post #215
ot wrote: post #185

Earlier quoted context omitted.

It's not just that zeroing got cheaper, but also we're doing a lot less of it, because jemalloc got much better. If the allocator returns a page to the kernel and then immediately asks back for one, it's not doing its job well: the main purpose of the allocator is to cache allocations from the kernel. Those patches are pre-decay, pre-background purging thread; these changes significantly improve how jemalloc holds on…

I am trying to understand the reason behind why "zeroing got cheaper" circa 2012-2014. Do you have some plausible explanations that you can share? Haswell (2013) doubled the store throughput to 32 bytes/cycle per core, and Sandy Bridge (2011) doubled the load throughput to the same, but the dataset being operated at FB is most likely much larger than what L1+L2+L3 can fit so I am wondering how much effect the vectori…

My memory is that Ivy Bridge was when it started being different.

Re: Meta’s renewed commitment to jemalloc

#223

Post by sumtechguy »

ot wrote: post #186
carlos256 wrote: post #100

If you need to optimize the allocator you are doing it wrong.

That's a false dichotomy: you optimize both the application and the allocator. A 0.5% improvement may not be a lot to you, but at hyperscaler scale it's well worth staffing a team to work on it, with the added benefit of having people on hand that can investigate subtle bugs and pathological perf behaviors.

exactly. I can think of at least 5 different projects I have been on where a better allocator would made a world of difference. I can also think of another 5 where it probably would have been a waste of time to even fiddle with.

but as usual there is an xkcd for that. https://xkcd.com/1205/

One project I spent a bunch of time optimizing the write path of I/O. It was just using standard fwrite. But by staging items correctly it was an easy 10x speed win. Those optimizations sometimes stack up and count big. But it also had a few edges on it, so use with care.

Re: Meta’s renewed commitment to jemalloc

#224

Post by pjmlp »

cogman10 wrote: post #218
pjmlp wrote: post #192

Earlier quoted context omitted.

Depends on which JVM, PTC and Aicas do alright with their real time GCs for embedded deployment.

I've never really used anything other than the OpenJDK and Azuls. How does PTC and Aicas does GC? Is it ref counted? I'm guessing they aren't doing moving collectors.

They are real time GCs, nothing to do with refcounting.

One of the founding members of Aicas is the author of "Hard Realtime Garbage Collection in Modern Object Oriented Programming Languages" book, which was done as part of his PhD.

Re: Meta’s renewed commitment to jemalloc

#225

Post by ahoka »

menaerus wrote: post #215
ot wrote: post #185

Earlier quoted context omitted.

It's not just that zeroing got cheaper, but also we're doing a lot less of it, because jemalloc got much better. If the allocator returns a page to the kernel and then immediately asks back for one, it's not doing its job well: the main purpose of the allocator is to cache allocations from the kernel. Those patches are pre-decay, pre-background purging thread; these changes significantly improve how jemalloc holds on…

I am trying to understand the reason behind why "zeroing got cheaper" circa 2012-2014. Do you have some plausible explanations that you can share? Haswell (2013) doubled the store throughput to 32 bytes/cycle per core, and Sandy Bridge (2011) doubled the load throughput to the same, but the dataset being operated at FB is most likely much larger than what L1+L2+L3 can fit so I am wondering how much effect the vectori…

AVX maybe?

Re: Meta’s renewed commitment to jemalloc

#227

Post by gzread »

pabs3 wrote: post #169
gzread wrote: post #168

Earlier quoted context omitted.

I don't remember. It was years ago.

Probably SerNet or one of the Samba Commercial Support companies like Catalyst: https://www.samba.org/samba/support/globalsupport.html

Might've been Catalyst. I see it's a NZ company not Australia.

Re: Meta’s renewed commitment to jemalloc

#228

Post by CyberDildonics »

cogman10 wrote: post #217
CyberDildonics wrote: post #154

Earlier quoted context omitted.

I'm not sure why you're rationale for how to deal with garbage collected memory is based on a guy that didn't know standard data structures and your own gut feelings. Any program that cares about performance is going to focus on minimizing memory allocation first. The difference between a GCed language like java is that the problems manifest as gc pauses that may or may not be predictable. In a language like C++ you…

Well, let me just circle back to the start of this comment chain. > Many programs in GC language end up fighting the GC by allocating a large buffer and managing it by hand That's the primary thing I'm contending with. This is a strategy for fighting the GC, but it's also generally a bad strategy. One that I think gets pulled more because someone heard of the suggestion and less because it's a good way to make things…

This is a strategy for fighting the GC, but it's also generally a bad strategy.

Allocating a large buffer is literally what an array or vector is. A heap uses a heap structure and hops around in memory for every allocation and free. It gets worse the more allocations there are. The allocations are fragmented and in different parts of memory.

Allocating a large buffer takes care of all this if it is possible to anything else. It doesn't make sense to make lots of heap allocations when what you want is multiple items next to each other in memory and one heap allocation.

That guy I'm talking about did a lot of "performance optimizations" based on gut feelings and not data.

You need to let this go, that guy has nothing to do with what works when optimizing memory usage and allocation.

But I've further observed that when it comes to optimizing for the GC, a large amount of problems don't need such an extreme measure like building your own memory buffer and managing it directly.

Making an array of contiguous items is not an "extreme strategy", it's the most efficient and simplest way for a program to run. Other memory allocations can just be an extension of this.

I agree that many programs with a GC will probably need to change their algorithms to minimize allocations. I disagree that "allocating a large buffer and managing it by hand"

If you need the same amount of memory but need to minimize allocations how do you think that is done? You make larger allocations and split them up. You keep saying "managing it by hand" as if there is something that has to be tricky or difficult. Using indices of an array is not difficult and neither is handing out indices or ranges to in small sections.

Re: Meta’s renewed commitment to jemalloc

#229

Post by adsharma »

teiferer wrote: post #220
debo_ wrote: post #199

Earlier quoted context omitted.

This is literally how pretty much every conversation goes when you work with people close to the metal. It's a stylistic thing at this point. For what it's worth, 20 years ago all programming newsgroups were like this. I grew my thick skin on alt.lang.perl lol

Except one is an employee and the other one is an ex employee. The bias this introduces is not just a minor nuance, it's what fuels the public conflict and causes everybody else to double check their popcorn reserves. Of course technical discussions happen all the time at companies between competent people. But you don't do that in public, nor is this a technical debate: "I don't recall talking to you about it" - "I…

Important distinction yes. It also means I can't go back and check the thread on what was said and when. Nor do I want to.

Always good to talk face to face if you're have strong feelings about something. When I said "talk" I meant literally face to face.

Spending a decade or so on lkml, everyone develops a thick skin. But mix it with the corporate environment, Facebook 2011, being an ex-employee adds more to the drama.

Having read through the comments here, I'm still of the opinion that any HW changes had a secondary effect and the primary contributor was a change in how HHVM/jemalloc interacted with MADV.

One more suggestion: evaluate more than one app and company wide profiling data to make such decisions.

One of the challenges in doing so is the large contingent of people who don't have an understanding of CPU uarch/counters and yet have a negative opinion of their usefulness to make decisions like this.

So the only tool you have left with is to run large scale rack level tests in a close to prod env, which has its own set of problems and benefits.

Re: Meta’s renewed commitment to jemalloc

#230

Post by jeffbee »

menaerus wrote: post #216
jeffbee wrote: post #123

Earlier quoted context omitted.

That strikes me as a common hugepages win. People never believe you, though, when you say you can make their thing 20% faster for free.

Then it should be pretty easy to display that 20% "faster for free", no? But as always the devil is in the details. I experimented a lot with huge pages, and although in theory you should see the performance boost, the workloads I have been using to test this hypothesis did not end up with anything statistically significant/measurable. So, my conclusion was ... it depends.

Of course, it only helps workloads that exhibit high rates of page table walking per instruction. But those are really common.
Post reply on HN