Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Jemalloc Postmortem

jasone.github.io

Post reply on HN
Print view View original

241–250 of 250 posts

Re: Jemalloc Postmortem

#241

Post by vlovich123 »

jeffbee wrote: post #68
vlovich123 wrote: post #60

Earlier quoted context omitted.

The “system” allocator is managing memory within a process boundary. The kernel is responsible for managing it across processes. Claiming that a user space allocator is greedily inefficient is voodoo reasoning that suggests the person making the claim has a poor grasp of architecture.

There are shared resources involved though, for example one process can cause a lot of traffic in khugepaged. However I would point out that is an endemic risk of Linux's overall architecture. Any process can cause chaos by dirtying pages, or otherwise triggering reclaim.

That’s generally true of any allocator and assuming glibc’s behavior would help mitigate this is critically not something kernel engineers design around nor something glibc allocator is trying to achieve as a design goal.

Re: Jemalloc Postmortem

#242

Post by vlovich123 »

jeffbee wrote: post #155
nicoburns wrote: post #120

Earlier quoted context omitted.

That seems odd though, seeing as this is one of the main criticisms of glibc's allocator.

In the containerized environments where these allocators were mainly developed, it is all but totally pointless to return memory to the kernel. You might as well keep everything your container is entitled to use, because it's not like the other containers can use it. Someone or some automatic system has written down how much memory the container is going to use.

glibc is not written in a containerized environment and I personally think it’s telling that a core feature of the more recent tcmalloc Google open sourced is that it returns memory efficiently, so clearly even in containerized environments it’s important. The reason for this is how kernels deal with compressing pages and pages released to the kernel are explicitly zeroed (unlike the user space allocator) which aids in the efficiency of the compression even in a containerized workload because those pages can just be skipped since they’re unused and the kernel can share the reference zeroed page for lazy allocations.

Also the kernel itself has memory needs for lots of things and it not having memory or having to go on a hunt to find contiguous pages is not good. Additionally in a VM or container environment there’s other containers and VMs running on that machine so the memory will also eventually get percolated up to the hyper visor to rebalance. None of this happens if the user space allocator hangs on to memory needlessly in a greedy fashion and indeed such an application would be more subject to the OOM killer.

Re: Jemalloc Postmortem

#243

Post by vlovich123 »

toast0 wrote: post #180
jeffbee wrote: post #155

Earlier quoted context omitted.

In the containerized environments where these allocators were mainly developed, it is all but totally pointless to return memory to the kernel. You might as well keep everything your container is entitled to use, because it's not like the other containers can use it. Someone or some automatic system has written down how much memory the container is going to use.

Returning no longer used anonymous memory is not without benefits. Returning pages allows them to be used for disk cache. They can be zeroed in the background by the kernel which may save time when they're needed again, or zeroing can be avoided if the kernel uses them as the destination of a full page DMA write. Also, returning no longer used pages helps get closer to a useful memory used measurement. Measuring memo…

Zeroed pages also compress more efficiently because the compressor doesn’t actually need to process them.

Re: Jemalloc Postmortem

#244

Post by alextingle »

mort96 wrote: post #126
alextingle wrote: post #116

Earlier quoted context omitted.

>> `#include "base/pc.h"`, where that `"base/pc.h"` path is not relative to the file doing the include. > I have to disagree on this one. The double-quotes literally mean "this dependency is relative to the current file". If you want to depend on a -I, then signal that by using angle brackets.

Eh, no. The quotes mean "this is not a dependency on a system library". Quotes can include relative to the files, or they can include things relative to directories specified with -I. The only thing they can't is include things relative to directories specified with -isystem and system include directories. I would be surprised if I read some project's code where angle brackets are used to include headers from within…

The only difference between "" and is that the former adds the current file's directory to the beginning of the search path.

So the only reason to use "" instead of is when you need that behaviour, because the dependency is relative to the current file.

If you use "" in any other situation, then you are introducing a potential error, because now someone can change the meaning of your code simply by creating a file with a name and location that happens to match your dependency.

(Yes, some compilers have -isystem and -iquote which modify that behaviour, but those options are not standard, and can't be relied upon. I'd strongly advise against their use.)

Re: Jemalloc Postmortem

#245

Post by nixgeek »

jeffbee wrote: post #93
lmm wrote: post #88

Earlier quoted context omitted.

> Both jemalloc and tcmalloc evolved and were refined in antagonistic multitenant environments without one overwhelming application. They are optimal for that exact thing. They were mostly optimised on Facebook/Google server-side systems, which were likely one application per VM, no? (Unlike desktop usage where users want several applications to run cooperatively). Firefox is a different case but apparently mainline…

Google runs dozens to hundreds of unrelated workloads in lightweight containers on a single machine, in "borg". Facebook has a thing called "tupperware" with the same property.

I think Tupperware was rebranded to Twine sometime about 6-7 years ago.

Re: Jemalloc Postmortem

#246

Post by Crespyl »

mort96 wrote: post #106
Crespyl wrote: post #73

Earlier quoted context omitted.

Can you elaborate on this? I don't know much about allocators. How would the allocator know that some block is unused, short of `free` being called? Does glibc not return all memory after a `free`? Do other allocators do something clever to automatically release things? Is there just a lot of bookkeeping overhead that some allocators are better at handling?

They're not really correct, glibc will return stuff back to the OS. It just has some quirks about how and when it does it. First, some background: no allocator will return memory back to the kernel for every `free`. That's for performance and memory consumption reasons: the smallest unit of memory you can request from and return to the kernel is a page (typically 4kiB or 16kiB), and requesting and returning memory (t…

Thank you for the rundown, that was very informative. I love learning about these sorts of internals.

Re: Jemalloc Postmortem

#247

Post by jcupitt »

tecleandor wrote: post #111
thewisenerd wrote: post #94

Earlier quoted context omitted.

indeed! most image processing golang services suggest/use jemalloc the top 3 from https://github.com/topics/resize-images (as of 2025-06-13) imaginary: https://github.com/h2non/imaginary/blob/1d4e251cfcd58ea66f83... imgproxy: https://web.archive.org/web/20210412004544/https://docs.imgp... (linked from a discussion in the imaginary repo) imagor: https://github.com/cshum/imagor/blob/f6673fa6656ee8ef17728f2...

Yep, imgproxy seems to use libvips, that recommends jemalloc. I was checking and this is a funny (not) bug report: https://github.com/libvips/libvips/discussions/3019

Hello, libvips author here. This is probably the canonical thread about libvips and memory fragmentation, and the funniest graph:

https://github.com/lovell/sharp/issues/955#issuecomment-5458...

(that specific graph is for switching from glib to the musl memory allocator, but jemalloc gives a very similar result)

Re: Jemalloc Postmortem

#248

Post by jcupitt »

thewisenerd wrote: post #94
adityapatadia wrote: post #84

Jason, here is a story about how much your work impacts us. We run a decently sized company that processes hundreds of millions of images/videos per day. When we first started about 5 years ago, we spent countless hours debugging issues related to memory fragmentation. One fine day, we discovered Jemalloc and put it in our service, which was causing a lot of memory fragmentation. We did not think that those 2 lines o…

indeed! most image processing golang services suggest/use jemalloc the top 3 from https://github.com/topics/resize-images (as of 2025-06-13) imaginary: https://github.com/h2non/imaginary/blob/1d4e251cfcd58ea66f83... imgproxy: https://web.archive.org/web/20210412004544/https://docs.imgp... (linked from a discussion in the imaginary repo) imagor: https://github.com/cshum/imagor/blob/f6673fa6656ee8ef17728f2...

Those three all use libvips as the image processing engine, fwiw, so it's maybe not a very wide survey.

libvips is fairly highly threaded and does a lot of alloc/free, so it's challenging for most heap implementations.

Re: Jemalloc Postmortem

#249

Post by [deleted] »

jeffbee wrote: post #238
atombender wrote: post #237

Earlier quoted context omitted.

I know Google has good engineering, but I find this a bit implausible? For most applications, especially request/response type apps like web servers, "right sizing" truly correctly while accounting for spikes takes a lot of engineering effort to fully account for how much allocation a single request will need, then ensuring the maximum concurrent requests never go beyond that so you never risk OOMs. I can see this be…

The reason I hedged and said "... or some automatic system ..." was because they use a machine-learned forecast of the memory requirements of every container and use that as the soft limit for the container when it starts. You can read about that at [1]. But what I was getting at is that using less than the configured amount of memory does not lead to more containers able to be scheduled on a given machine, nor does…

[deleted]

Re: Jemalloc Postmortem

#250

Post by adityapatadia »

jcupitt wrote: post #248
thewisenerd wrote: post #94

Earlier quoted context omitted.

indeed! most image processing golang services suggest/use jemalloc the top 3 from https://github.com/topics/resize-images (as of 2025-06-13) imaginary: https://github.com/h2non/imaginary/blob/1d4e251cfcd58ea66f83... imgproxy: https://web.archive.org/web/20210412004544/https://docs.imgp... (linked from a discussion in the imaginary repo) imagor: https://github.com/cshum/imagor/blob/f6673fa6656ee8ef17728f2...

Those three all use libvips as the image processing engine, fwiw, so it's maybe not a very wide survey. libvips is fairly highly threaded and does a lot of alloc/free, so it's challenging for most heap implementations.

We use libvips as well sir. We can't overstate how much we benefit from your work!
Post reply on HN