Meta’s renewed commitment to jemalloc • Hacker News Board

Re: Meta’s renewed commitment to jemalloc

#111

Post by codexon » Mon, Mar 16, 2026, 9:17 PM UTC

sylware wrote: post #12

bmenrigh wrote: post #8

I recently started using Microsoft's mimalloc (via an LD_PRELOAD) to better use huge (1 GB) pages in a memory intensive program. The performance gains are significant (around 20%). It feels rather strange using an open source MS library for performance on my Linux system. There needs to be more competition in the malloc space. Between various huge page sizes and transparent huge pages, there are a lot of gains to be…

If there is so much performance difference among generic allocators, it means you need semantic optimized allocators (unless performance is actually not that much important in the end).

Agreed mostly. Going from standard library to something like jemalloc or tcmalloc will give you around 5-10% wins which can be significant, but the difference between those generic allocators seem small. I just made a slab allocator recently for a custom data type and got speedups of 100% over malloc.

Re: Meta’s renewed commitment to jemalloc

#112

Post by codexon » Mon, Mar 16, 2026, 9:20 PM UTC

skavi wrote: post #47

bmenrigh wrote: post #8

I recently started using Microsoft's mimalloc (via an LD_PRELOAD) to better use huge (1 GB) pages in a memory intensive program. The performance gains are significant (around 20%). It feels rather strange using an open source MS library for performance on my Linux system. There needs to be more competition in the malloc space. Between various huge page sizes and transparent huge pages, there are a lot of gains to be…

We evaluated a few allocators for some of our Linux apps and found (modern) tcmalloc to consistently win in time and space. Our applications are primarily written in Rust and the allocators were linked in statically (except for glibc). Unfortunately I didn't capture much context on the allocation patterns. I think in general the apps allocate and deallocate at a higher rate than most Rust apps (or more than I'd like…

This is similar to what I experienced when I tested mimalloc many years ago. If it was faster, it wasn't faster by much, and had pretty bad worst cases.

Re: Meta’s renewed commitment to jemalloc

#113

Post by sethhochberg » Mon, Mar 16, 2026, 9:25 PM UTC

bluGill wrote: post #60

bfgeek wrote: post #11

One has to wonder if this due to the global memory shortage. ("Oh - changing our memory allocator to be more efficient will yield $XXM dollar savings over the next year").

Facebook had talks already years ago (10+) - nobody was allowed to share real numbers, but several facebook employed where allowed to share that the company has measured savings from optimizations. Reading between the lines, a 0.1% efficiency improvement to some parts of Facebook would save them $100,000 a month (again real numbers were never publicly shared so there is a range - it can't be less than $20,000), and s…

I don't have many regrets about having spent my career in (relatively) tiny companies by comparison, but it sure does sound fun to be on the other side for this kind of thing - the scale where micro-optimizations have macro impact.

In startups I've put more effort into squeezing blood from a stone for far less change; even if the change was proportionally more significant to the business. Sometimes it would be neat to say "something I did saved $X million dollars or saved Y kWh of energy" or whatever.

Re: Meta’s renewed commitment to jemalloc

#114

Post by jshorty » Mon, Mar 16, 2026, 9:34 PM UTC

twodave wrote: post #96

jshorty wrote: post #83

Surprised not to see any mention of the global memory supply shock. Would love to learn more about how that economic is shifting software priorities toward memory allocation for the first time in my (relatively young) career

While it may seem directly related, it's just not. These things are worked on regardless of how cheap or expensive RAM is, because optimizing memory footprint pretty much always leads to fewer machines leased, which is a worthwhile goal even for smaller shops.

That's useful to know, thank you.

Re: Meta’s renewed commitment to jemalloc

#115

Post by cbarrick » Mon, Mar 16, 2026, 9:43 PM UTC

carlos256 wrote: post #100

If you need to optimize the allocator you are doing it wrong.

Exactly. No need to engineer an allocator. You only live once!

    void* malloc(size_t size) {
        void *ptr = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_ANON, -1, 0);
        return (ptr == MAP_FAILED) ? NULL : ptr;
    }

    void free(void *ptr) { /* YOLO */ }

/s

Re: Meta’s renewed commitment to jemalloc

#116

Post by Sesse__ » Mon, Mar 16, 2026, 9:44 PM UTC

m463 wrote: post #68

bmenrigh wrote: post #8

I recently started using Microsoft's mimalloc (via an LD_PRELOAD) to better use huge (1 GB) pages in a memory intensive program. The performance gains are significant (around 20%). It feels rather strange using an open source MS library for performance on my Linux system. There needs to be more competition in the malloc space. Between various huge page sizes and transparent huge pages, there are a lot of gains to be…

I remember in the early days of web services, using the apache portable runtime, specifically memory pools. If you got a web request, you could allocate a memory pool for it, then you would do all your memory allocations from that pool. And when your web request ended - either cleanly or with a hundred different kinds of errors, you could just free the entire pool. it was nice and made an impression on me. I think th…

This is called “an arena” more generally, and it is in wide use across many forms of servers, compilers, and others.

Re: Meta’s renewed commitment to jemalloc

#117

Post by apatheticonion » Mon, Mar 16, 2026, 10:01 PM UTC

As an Australian who was just made redundant from a role that involved this type of low level programming - I love working on these these kinds of challenges.

I'm saddened that the job market in Australia is largely React CRUD applications and that it's unlikely I will find a role that lets me leverage my niche skill set (which is also my hobby)

Re: Meta’s renewed commitment to jemalloc

#118

Post by CyberDildonics » Mon, Mar 16, 2026, 10:06 PM UTC

cogman10 wrote: post #106

CyberDildonics wrote: post #105

Earlier quoted context omitted.

If people didn't need to do it, they wouldn't generally do it. Not always, but generally.

People do stuff they shouldn't all the time. For example, some code I had to clean up pretty early on in my career was a dev, for unknown reasons, reinventing the `ArrayList` and then using that invention as a set (doing deduplication by iterating over the elements and checking for duplicates). It was done in the name of performance, but it was never a slow part of the code. I replaced the whole thing with a `HashSet…

Reinventing data structures poorly is very common.

Heap allocation in java is something trivial happens constantly. People typically do funky stuff with memory allocation because they have to, because the GC is causing pauses.

People avoid system allocators in C++ too, they just don't have to do it because of uncontrollable pauses.

Re: Meta’s renewed commitment to jemalloc

#119

Post by Anon1096 » Mon, Mar 16, 2026, 10:08 PM UTC

bluGill wrote: post #60

bfgeek wrote: post #11

One has to wonder if this due to the global memory shortage. ("Oh - changing our memory allocator to be more efficient will yield $XXM dollar savings over the next year").

Facebook had talks already years ago (10+) - nobody was allowed to share real numbers, but several facebook employed where allowed to share that the company has measured savings from optimizations. Reading between the lines, a 0.1% efficiency improvement to some parts of Facebook would save them $100,000 a month (again real numbers were never publicly shared so there is a range - it can't be less than $20,000), and s…

I've worked on optimizing systems in that ballpark range, memory is worth saving but it isn't necessarily 1:1 with increasing revenue like CPU is. For CPU we have tables to calculate the infra cost savings (we're not really going to free up the server, more like the system is self balancing so it can run harder with the freed CPU), but for memory as long as we can load in whatever we want to (rec systems or ai models) we're in the clear so the marginal headroom isn't as important. It's more of a side thing that people optimizing CPU also get wins in by chance because the skillsets are similar.

Re: Meta’s renewed commitment to jemalloc

#120

Post by kenferry » Mon, Mar 16, 2026, 10:22 PM UTC

rvz wrote: post #59

vintermann wrote: post #29

Earlier quoted context omitted.

"Were I to reengage, the first step would be at least hundreds of hours of refactoring to pay off accrued technical debt." Facebook's coding AIs to the rescue, maybe? I wonder how good all these "agentic" AIs are at dreaded refactoring jobs like these.

> Facebook's coding AIs to the rescue, maybe? I wonder how good all these "agentic" AIs are at dreaded refactoring jobs like these. No. This is something you shouldn't allow coding agents anywhere near, unless you have expert-level understanding required to maintain the project like the previous authors have done without an AI for years.

Hm, I wonder.

I've done some work in this sort of area before, though not literally on a malloc. Yes you very much want to be careful, but ultimately it's the tests that give you confidence. Pound the heck out of it in multithreaded contexts and test for consistency.