Seems like they’d want to wait to commit until after the layoffs, right?
It's just one team with like 4 people. They can layoff a lot of staff from Metaverse.
Meta’s renewed commitment to jemalloc
71–80 of 259 posts
Re: Meta’s renewed commitment to jemalloc
#72> We plan to deliver improvements to [..] purging mechanisms
During my time at Facebook, I maintained a bunch of kernel patches to improve jemalloc purging mechanisms. It wasn't popular in the kernel or the security community, but it was more efficient on benchmarks for sure.
Many programs run multiple threads, allocate in one and free in the other. Jemalloc's primary mechanism used to be: madvise the page back to the kernel and then have it allocate it in another thread's pool.
One problem: this involves zero'ing memory, which has an impact on cache locality and over all app performance. It's completely unnecessary if the page is being recirculated within the same security domain.
The problem was getting everyone to agree on what that security domain is, even if the mechanism was opt-in.
Re: Meta’s renewed commitment to jemalloc
#73Earlier quoted context omitted.
You really need to benchmark your workloads, ideally with the "big 3" (jemalloc, tcmalloc, mimalloc). They all have their strengths and weaknesses. Jemalloc can usually keep the smallest memory footprint, followed by tcmalloc. Mimalloc can really speed things up sometimes. As usually, YMMV.
I've benchmarked them every few years, they never seem to differ by more than a few percent, and jemalloc seems to fragment and leak the least for processes running for months. Mimalloc made the claim that they were the fastest/best when they released and that didn't hold up to real world testing, so I am not inclined to trust it now.
> Mimalloc made the claim that they were the fastest/best when they released and that didn't hold up to real world testing
That’s… ahistorical, at least so far as I remember. It wasn’t marketed as either of those; it was marketed as small/simple/consistent with an opt-in high-severity mode, and then its performance bore out as a result of the first set of target features/design goals. It was mainly pushed as easy to adopt, easy to use, easy to statically link, etc.
Re: Meta’s renewed commitment to jemalloc
#74(wrong thread)
The URL of this story seems to have changed to a Meta press release. What are you quoting?
Re: Meta’s renewed commitment to jemalloc
#75Earlier quoted context omitted.
One of the best parts about GC languages is they tend to have much more efficient allocation/freeing because the cost is much more lumped together so it shows up better in a profile.
When it works. Many programs in GC language end up fighting the GC by allocating a large buffer and managing it by hand anyway because when performance counts you can't have allocation time in there at all. (you see this in C all the time as well)
That's generally a bad idea. Not always, but generally.
It was a better idea when Java had the old mark and sweep collector. However, with the generational collectors (which are all Java collectors now. except for epsilon) it's more problematic. Reusing buffers and objects in those buffers will pretty much guarantees that buffer ends up in oldgen. That means to clear it out, the VM has to do more expensive collections.
The actual allocation time for most of Java's collectors is almost 0, it's a capacity check and a pointer bump in most circumstances. Giving the JVM more memory will generally solve issues with memory pressure and GC times. That's (generally) a better solution to performance problems vs doing the large buffer.
Now, that said, there certainly have been times where allocation pressure is a major problem and removing the allocation is the solution. In particular, I've found boxing to often be a major cause of performance problems.
Re: Meta’s renewed commitment to jemalloc
#76One has to wonder if this due to the global memory shortage. ("Oh - changing our memory allocator to be more efficient will yield $XXM dollar savings over the next year").
Yeah, identifying single-digit millions of savings out of profiles is relatively common practice at Meta. It's ~easy to come up with a big number when the impact is scaled across a very large numbers of servers. There is a culture of measuring and documenting these quantified wins.
Re: Meta’s renewed commitment to jemalloc
#77Earlier quoted context omitted.
What do you mean - if Java returns memory to the OS? Which one - Java heap of the malloc/free by the JVM?
Java is pretty greedy with the memory it claims. Especially historically it was pretty hard to get the JVM to release memory back to the OS. To an outsider, that looks like the JVM heap just steadily growing, which is easy to mistake for a memory leak.
Java has a quite strict max heap setting, it's very uncommon to let it allocate up to 25% of the system memory (the default). It won't grow past that point, though.
Baring bugs/native leaks - Java has a very predictable memory allocation.
Re: Meta’s renewed commitment to jemalloc
#78Earlier quoted context omitted.
Meta still maintained it and actively pushed commits to it fixing bugs and adding improvements. From this blog post it sounds like they are increasing investment into it along with resurrecting the original repo. When the repo was archived Meta said that development on jemalloc would be focused towards Meta's own goals and needs as opposed to the larger ecosystem.
I'm not directly involved enough to dig into the details here, but facebook/jemalloc currently says: > This branch is 71 commits ahead of and 70 commits behind jemalloc/jemalloc:dev. It looks like both have been independently updated.
This looks a lot as if the facebook/jemalloc repo inserted a single commit 70 commits ago and then rebased the changes in the original repo on top. Because the commit SHAs for the changes pulled in change you see this result.
Re: Meta’s renewed commitment to jemalloc
#79Earlier quoted context omitted.
What do you mean - if Java returns memory to the OS? Which one - Java heap of the malloc/free by the JVM?
Java is pretty greedy with the memory it claims. Especially historically it was pretty hard to get the JVM to release memory back to the OS. To an outsider, that looks like the JVM heap just steadily growing, which is easy to mistake for a memory leak.
> Especially historically it was pretty hard to get the JVM to release memory back to the OS.
This feels like a huge understatement. I still have some PTSD around when I did Java professionally between like 2005 and 2014.
The early part of that was particularly horrible.
Re: Meta’s renewed commitment to jemalloc
#80One has to wonder if this due to the global memory shortage. ("Oh - changing our memory allocator to be more efficient will yield $XXM dollar savings over the next year").
Oooh maybe finally time for lovingly hand-optimized assembly to come back in fashion! (It probably has in AI workloads or so I daydream)