Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jemalloc Postmortem

jasone.github.io

Post reply on HN
Print view View original

231–240 of 250 posts

Re: Jemalloc Postmortem

#231

Post by foldr »

Twirrim wrote: post #69
jeffbee wrote: post #7

Earlier quoted context omitted.

For an example of why an allocator is a maintenance treadmill, consider that C++ recently (relatively) added sized delete, and Linux recently gained transparent huge pages.

It's been 14 years since THP got added to the kernel[1], surely we're past calling that "recent" :) https://www.kernelconfig.io/config_transparent_hugepage

> In particular, the seeds for principled huge page allocation (HPA) were sown way back in 2016! HPA work continued apace for several years, slowed, then stagnated as tweaks piled on top of each other without the requisite refactoring that keeps a codebase healthy. This feature trajectory recently cratered.

Re: Jemalloc Postmortem

#232

Post by procaryote »

nh2 wrote: post #124
masklinn wrote: post #70

Earlier quoted context omitted.

glibc will return memory to the OS just fine, the problem is that its arena design is extremely prone to fragmentation, so you end up with a bunch of arenas which are almost but not quite empty and can't be released, but can’t really be used either. In fact, Jason himself (the author of jemalloc and TFA) posted an article on glibc malloc fragmentation 15 years ago: https://web.archive.org/web/20160417080412/http://ww…

glibc does NOT return memory to the OS just fine. In my experience it delays it way too much, causing memory overuse and OOMs. I have a Python program that allocates 100 GB for some work, free()s it, and then calls a subprocess that takes 100 GB as well. Because the memory use is serial, it should fit in 128 GB just fine. But it gets OOM-killed, because glibc does not turn the free() into an munmap() before the subpr…

You could do the memory heavy python part in a separate process as well. That removes the need to depend on quirks of the allocator

Re: Jemalloc Postmortem

#233

Post by procaryote »

kstrauser wrote: post #44
skeptrune wrote: post #23

Kind of nuts that he worked on Jemalloc for over a decade while having personal preference for garbage collection. I'm surprised he doesn't have more regret.

Why are those two mutually exclusive? I'd think that a high performance allocator would be especially crucial in the implementation of a fast garbage collected language. For example, in Python you can't alloc(n * sizeof(obj)) to reserve that much contiguous space for n objects. Instead, you use the builtins which isolate you from that low-level bookkeeping. Those builtins have to be pretty fast or performance would b…

Python performance is terrible though...

Re: Jemalloc Postmortem

#234

Post by michaelcampbell »

Svetlitski wrote: post #36

I understand the decision to archive the upstream repo; as of when I left Meta, we (i.e. the Jemalloc team) weren’t really in a great place to respond to all the random GitHub issues people would file (my favorite was the time someone filed an issue because our test suite didn’t pass on Itanium lol). Still, it makes me sad to see. Jemalloc is still IMO the best-performing general-purpose malloc implementation that’s…

> filed an issue because our test suite didn’t pass on Itanium lol

For the non low-level programmers in the bowels of memory allocators among us, why is this a "lol"?

Re: Jemalloc Postmortem

#235

Post by Svetlitski »

michaelcampbell wrote: post #234
Svetlitski wrote: post #36

I understand the decision to archive the upstream repo; as of when I left Meta, we (i.e. the Jemalloc team) weren’t really in a great place to respond to all the random GitHub issues people would file (my favorite was the time someone filed an issue because our test suite didn’t pass on Itanium lol). Still, it makes me sad to see. Jemalloc is still IMO the best-performing general-purpose malloc implementation that’s…

> filed an issue because our test suite didn’t pass on Itanium lol For the non low-level programmers in the bowels of memory allocators among us, why is this a "lol"?

The Itanium ISA was an infamous failure, never seeing widespread usage, hence people often referring to it as “The Itanic” (i.e. the much-touted ship that immediately sunk). The fact that anyone would be using it today at all is sort of hilariously niche, and is illustrative of how wide-ranging and obscure the issues filed to the GitHub repo could be. On a similar token I recall seeing an issue (or maybe it was a PR?) to fix our build on GNU Herd.

Re: Jemalloc Postmortem

#236

Post by ComputerGuru »

ahartmetz wrote: post #200
Iwan-Zotow wrote: post #188

Earlier quoted context omitted.

+1 windows def allocator is pos. Jemalloc rules

>windows def allocator is pos Wow, still? I remember allocator benchmarks from 10-15 years ago where there were some notable differences between allocators... and then Windows with like 20% the performance of everything else!

It’s improved considerably since.

Re: Jemalloc Postmortem

#237

Post by atombender »

jeffbee wrote: post #155
nicoburns wrote: post #120

Earlier quoted context omitted.

That seems odd though, seeing as this is one of the main criticisms of glibc's allocator.

In the containerized environments where these allocators were mainly developed, it is all but totally pointless to return memory to the kernel. You might as well keep everything your container is entitled to use, because it's not like the other containers can use it. Someone or some automatic system has written down how much memory the container is going to use.

I know Google has good engineering, but I find this a bit implausible?

For most applications, especially request/response type apps like web servers, "right sizing" truly correctly while accounting for spikes takes a lot of engineering effort to fully account for how much allocation a single request will need, then ensuring the maximum concurrent requests never go beyond that so you never risk OOMs.

I can see this being fine-tuned for extremely high-scale, core services like load balancers, SDNs, file systems etc., where you probably want to allocate all your data structures at startup time and never actually allocate anything after that, and you probably have whole teams of engineers devoted to just single services. But not most apps?

Surely it's better for containers to share system memory, and rely on limits and resource-driven autoscaling to make the system resilient?

Re: Jemalloc Postmortem

#238

Post by jeffbee »

atombender wrote: post #237
jeffbee wrote: post #155

Earlier quoted context omitted.

In the containerized environments where these allocators were mainly developed, it is all but totally pointless to return memory to the kernel. You might as well keep everything your container is entitled to use, because it's not like the other containers can use it. Someone or some automatic system has written down how much memory the container is going to use.

I know Google has good engineering, but I find this a bit implausible? For most applications, especially request/response type apps like web servers, "right sizing" truly correctly while accounting for spikes takes a lot of engineering effort to fully account for how much allocation a single request will need, then ensuring the maximum concurrent requests never go beyond that so you never risk OOMs. I can see this be…

The reason I hedged and said "... or some automatic system ..." was because they use a machine-learned forecast of the memory requirements of every container and use that as the soft limit for the container when it starts. You can read about that at [1]. But what I was getting at is that using less than the configured amount of memory does not lead to more containers able to be scheduled on a given machine, nor does it lead to lower economic chargeback. Machines are scheduled and operators are charged by the configured limit, not the usage.

Giving memory back to the operating system is antithetical to the nature of caching allocators ("caching" is right there in the name of "tcmalloc"). The whole point of a caching allocator is that if you needed the memory once, you'll probably need it again, and most likely right now. At most what these allocators will do unless you configure them differently is to release memory to the system very, very slowly, and only if an entirely empty huge page — a contiguous area of several megabytes — surfaces. You can read how grudgingly the tcmalloc authors allow releasing at [2]. jemalloc was once pretty aggressive about releasing to the OS, but these days it is not. I think this reflects its evolution to suit Meta internal workloads, and increased understanding of the costs of releasing memory from a huge-page-aware allocator.

1: https://dl.acm.org/doi/pdf/10.1145/3342195.3387524 2: https://github.com/google/tcmalloc/blob/master/docs/tuning.m...

Re: Jemalloc Postmortem

#239

Post by vlovich123 »

mort96 wrote: post #106
Crespyl wrote: post #73

Earlier quoted context omitted.

Can you elaborate on this? I don't know much about allocators. How would the allocator know that some block is unused, short of `free` being called? Does glibc not return all memory after a `free`? Do other allocators do something clever to automatically release things? Is there just a lot of bookkeeping overhead that some allocators are better at handling?

They're not really correct, glibc will return stuff back to the OS. It just has some quirks about how and when it does it. First, some background: no allocator will return memory back to the kernel for every `free`. That's for performance and memory consumption reasons: the smallest unit of memory you can request from and return to the kernel is a page (typically 4kiB or 16kiB), and requesting and returning memory (t…

I think you’re right but that’s basically my point. In practice glibc effectively ends up not freeing very much memory back to the OS. That’s the same as if it didn’t even bother releasing even though in theory it does have a code path to free.

The kernel does have techniques to try to free up that memory anyway (compressing them) but critically it’s not as effective as just letting the kernel know those pages are unused.

Re: Jemalloc Postmortem

#240

Post by vlovich123 »

favorited wrote: post #76
vlovich123 wrote: post #60

Earlier quoted context omitted.

The “system” allocator is managing memory within a process boundary. The kernel is responsible for managing it across processes. Claiming that a user space allocator is greedily inefficient is voodoo reasoning that suggests the person making the claim has a poor grasp of architecture.

For context, the "allocator engineer" I was talking to was a kernel engineer - they have an extremely solid grasp of their platform's architecture. The whole advantage of being the platform's system allocator is that you can have a tighter relationship between the library function and the kernel implementation.

I’m not generally aware of any system allocator that’s written hand in glove with the kernel’s allocator or somehow interops better for overall system efficiency at the cost of behavior in-app. Care to provide an example?
Post reply on HN