71–80 of 250 posts
Earlier quoted context omitted.
For an example of why an allocator is a maintenance treadmill, consider that C++ recently (relatively) added sized delete, and Linux recently gained transparent huge pages.
It's been 14 years since THP got added to the kernel[1], surely we're past calling that "recent" :) https://www.kernelconfig.io/config_transparent_hugepage
Kind of nuts that he worked on Jemalloc for over a decade while having personal preference for garbage collection. I'm surprised he doesn't have more regret.
Switching to jemalloc instantly fixed an irksome memory leak in an embedded Linux appliance I inherited many moons ago. Thank you je, we salute you!
That’s because sane allocators that aren’t glibc will return unused memory periodically to the OS while glibc prefers to permanently retain said memory.
How would the allocator know that some block is unused, short of `free` being called? Does glibc not return all memory after a `free`? Do other allocators do something clever to automatically release things? Is there just a lot of bookkeeping overhead that some allocators are better at handling?
I understand the decision to archive the upstream repo; as of when I left Meta, we (i.e. the Jemalloc team) weren’t really in a great place to respond to all the random GitHub issues people would file (my favorite was the time someone filed an issue because our test suite didn’t pass on Itanium lol). Still, it makes me sad to see. Jemalloc is still IMO the best-performing general-purpose malloc implementation that’s…
That was me that filed the Itanium test suite failure. :)
A fellow traveler, ahoy!
Earlier quoted context omitted.
It's possible that they were referring to something specific about their platform and its system allocator, but like I said it was an anecdote about one engineer's statement. I just remember thinking it sounded fair at the time.
The “system” allocator is managing memory within a process boundary. The kernel is responsible for managing it across processes. Claiming that a user space allocator is greedily inefficient is voodoo reasoning that suggests the person making the claim has a poor grasp of architecture.
The whole advantage of being the platform's system allocator is that you can have a tighter relationship between the library function and the kernel implementation.
Nice post -- so does Facebook no longer use jemalloc at all? Or is it maintenance mode? Or I wonder if they could simply use tcmalloc or another allocator these days? Facebook infrastructure engineering reduced investment in core technology, instead emphasizing return on investment.
Your work was so impactful over a long period from Firefox to Facebook. Honored to have been a small part of it.
Though this was not the post I was expecting to show up today, it was super awesome for me to get to have played my tiny part in this big journey. Thanks for everything @je (and qi + david -- and all the contributors before and after my time!).
I’ve wondered about this before but never when around people who might know. From my outsider view, jemalloc looked like a strict improvement over glibc’s malloc, according to all the benchmarks I’d seen when the subject came up. So, why isn’t it the default allocator?
I will say, the Facebook people were very excited to share jemalloc with us when they acquired my employer, but we were using FreeBSD so we already had it and thought it was normal. :)
Oh that's interesting. jemalloc is the memory allocator used by redis, among other projects. Wonder what the performance impact will be if they have to change allocators.