Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta’s renewed commitment to jemalloc

engineering.fb.com

Post reply on HN
Print view View original

241–250 of 259 posts

Re: Meta’s renewed commitment to jemalloc

#241

Post by menaerus »

jeffbee wrote: post #230
menaerus wrote: post #216

Earlier quoted context omitted.

Then it should be pretty easy to display that 20% "faster for free", no? But as always the devil is in the details. I experimented a lot with huge pages, and although in theory you should see the performance boost, the workloads I have been using to test this hypothesis did not end up with anything statistically significant/measurable. So, my conclusion was ... it depends.

Of course, it only helps workloads that exhibit high rates of page table walking per instruction. But those are really common.

Yes, I understand that. It is implied that there's a high TLB miss rate. However, I'm wondering if the penalty which we can quantify as O(4) memory accesses for 4-level page table, which amounts to ~20 cycles if pages are already in L1 cache, or ~60-200 cycles if they are in L2/L3, would be noticeable in workloads which are IO bound. In other words, would such workloads benefit from switching to the huge pages when most of the time CPU anyways sits waiting on the data to arrive from the storage.

Re: Meta’s renewed commitment to jemalloc

#242

Post by genxy »

adsharma wrote: post #221
genxy wrote: post #178

Earlier quoted context omitted.

What metrics were improved by your patches?

Some more historical context. It wasn't a random optimization idea that I thought about in the shower and implemented the next day. Previous work on company wide profiling, where my contribution was low level perf_events plumbing: https://research.google/pubs/google-wide-profiling-a-continu... https://engineering.fb.com/2025/01/21/production-engineering... The profiling clearly showed kernel functions doing memzero a…

This kind of thing always struck me as something that the MMU and the memory controller could team up on. When you give back memory, you could not refresh it for some cycles. Or you could DMA the same page of zeros over all of it, so the CPU isn't involved in menial labor.

Re: Meta’s renewed commitment to jemalloc

#244

Post by jeffbee »

menaerus wrote: post #241
jeffbee wrote: post #230

Earlier quoted context omitted.

Of course, it only helps workloads that exhibit high rates of page table walking per instruction. But those are really common.

Yes, I understand that. It is implied that there's a high TLB miss rate. However, I'm wondering if the penalty which we can quantify as O(4) memory accesses for 4-level page table, which amounts to ~20 cycles if pages are already in L1 cache, or ~60-200 cycles if they are in L2/L3, would be noticeable in workloads which are IO bound. In other words, would such workloads benefit from switching to the huge pages when m…

In a multi-tenant environment, yes. The faster they can get off the CPU and yield to some other tenant, the better it is.

Re: Meta’s renewed commitment to jemalloc

#245

Post by Asm2D »

It would be great if Meta was able to sustain to support more open source projects, especially those they benefit from.

For example they use AsmJit in a lot of projects (both internal and open-source) and it's now unmaintained because of funding issues. Maybe they have now internal forks too.

Re: Meta’s renewed commitment to jemalloc

#246

Post by senderista »

jeffbee wrote: post #174
pram wrote: post #52

I used jemalloc recently for ComfyUI/Wan and it’s literally magic. I’m surprised it doesn’t come that way by default.

Allocators like that aren't the default for every process because they have higher startup costs. They are targeted to server workloads where startup cost doesn't matter, but it matters a lot if you're doing crud like starting millions of short-lived processes.

I don't think glibc malloc makes an optimal set of tradeoffs for any scenario.

Re: Meta’s renewed commitment to jemalloc

#247

Post by CyberDildonics »

cogman10 wrote: post #239
CyberDildonics wrote: post #235

Earlier quoted context omitted.

In the JVM, heap allocations are done via bump allocation. If that were true then they wouldn't be heap allocations. https://www.digitalocean.com/community/tutorials/java-jvm-me... https://docs.oracle.com/en/java/javase/21/core/heap-and-heap... not possible to do in the JVM, barring primitives Then you make data structures out of arrays of primitives. Easy to do? Sure. Easy to do fast? Well, no. That's entirely the r…

> If that were true then they wouldn't be heap allocations. "Heap" is a misnomer. It's not called that due to the classic CS "heap" datastructure. It's called that for the same reason it's called a heap allocation in C++. Modern C++ allocators don't use a heap structure either. How the JVM does allocations for all it's collectors is in fact a bump allocator in the heap space. There are some weedsy details (for exampl…

Modern C++ allocators don't use a heap structure either.

"Yes, malloc uses a heap data structure to allocate memory dynamically for programs. The heap allows for persistent memory allocation that can be managed manually by the programmer."

"How Malloc Works with the Heap

    Heap Data Structure: Malloc uses a heap data structure to manage memory. The heap is a region of a process's memory that is used for dynamic memory allocation.

    Memory Management: When you call malloc, it searches the heap for a suitable block of memory that can accommodate the requested size. If found, it allocates that memory and returns a pointer to it."
How the JVM does allocations for all it's collectors is in fact a bump allocator in the heap space.

This doesn't make sense. It's one or the other. A heap isn't about getting more memory or mapping it into a process space, it is about managing the memory already in the process space and being able to free memory in a different order than you allocated it, then give that memory back out without system calls.

https://www.geeksforgeeks.org/c/dynamic-memory-allocation-in...

https://en.wikipedia.org/wiki/C_dynamic_memory_allocation

JVM allocations are typically pointer bumps, adding a number to a register.

I think you are mixing up mapping memory into a process (which is a system call not a register addition) and managing the memory once it is in process space.

The allocator frees memory and reuses it within a process. If freeing it was as simple as subtracting from a register then there would be no difference in speed between the stack and the heap and there would be no GC pauses and no GC complexity. None of these things are true obviously since java has been dealing with these problems for 30 years.

This is why the JVM is so fast at allocation, much faster than C++ can be

Java is slower than C++ and less predictable because you can't avoid the GC which is the whole point here.

The original point was that you have to either avoid the GC or fight the GC and a lot of what you have talked about is either not true or explains why someone has to avoid and fight the GC in the first place.

Re: Meta’s renewed commitment to jemalloc

#248

Post by [deleted] »

CyberDildonics wrote: post #247
cogman10 wrote: post #239

Earlier quoted context omitted.

> If that were true then they wouldn't be heap allocations. "Heap" is a misnomer. It's not called that due to the classic CS "heap" datastructure. It's called that for the same reason it's called a heap allocation in C++. Modern C++ allocators don't use a heap structure either. How the JVM does allocations for all it's collectors is in fact a bump allocator in the heap space. There are some weedsy details (for exampl…

Modern C++ allocators don't use a heap structure either. "Yes, malloc uses a heap data structure to allocate memory dynamically for programs. The heap allows for persistent memory allocation that can be managed manually by the programmer." "How Malloc Works with the Heap Heap Data Structure: Malloc uses a heap data structure to manage memory. The heap is a region of a process's memory that is used for dynamic memory…

[deleted]

Re: Meta’s renewed commitment to jemalloc

#249

Post by adsharma »

genxy wrote: post #242
adsharma wrote: post #221

Earlier quoted context omitted.

Some more historical context. It wasn't a random optimization idea that I thought about in the shower and implemented the next day. Previous work on company wide profiling, where my contribution was low level perf_events plumbing: https://research.google/pubs/google-wide-profiling-a-continu... https://engineering.fb.com/2025/01/21/production-engineering... The profiling clearly showed kernel functions doing memzero a…

This kind of thing always struck me as something that the MMU and the memory controller could team up on. When you give back memory, you could not refresh it for some cycles. Or you could DMA the same page of zeros over all of it, so the CPU isn't involved in menial labor.

This is an old debate that goes back 25+ years. One of the differences in how Linux and FreeBSD handle the issue.

Linux developers believe that involving the CPU warms the caches and is a good thing.

Re: Meta’s renewed commitment to jemalloc

#250

Post by adgjlsfhk1 »

CyberDildonics wrote: post #247
cogman10 wrote: post #239

Earlier quoted context omitted.

> If that were true then they wouldn't be heap allocations. "Heap" is a misnomer. It's not called that due to the classic CS "heap" datastructure. It's called that for the same reason it's called a heap allocation in C++. Modern C++ allocators don't use a heap structure either. How the JVM does allocations for all it's collectors is in fact a bump allocator in the heap space. There are some weedsy details (for exampl…

Modern C++ allocators don't use a heap structure either. "Yes, malloc uses a heap data structure to allocate memory dynamically for programs. The heap allows for persistent memory allocation that can be managed manually by the programmer." "How Malloc Works with the Heap Heap Data Structure: Malloc uses a heap data structure to manage memory. The heap is a region of a process's memory that is used for dynamic memory…

You're wrong for like 6 different reasons.

Java does do bump pointer allocation. The key is that when GC runs, surviving objects get moved. The slow part of GC isn't the allocation (GCs generally have much faster allocators than malloc). The slow part is the barriers that the GC requires and the pauses.

Post reply on HN