Seems like Rust has made a solid choice in making jemalloc their default allocator.
Testing Memory Allocators: ptmalloc2 vs. tcmalloc vs. hoard vs. jemalloc
11–20 of 43 posts
Re: Testing Memory Allocators: ptmalloc2 vs. tcmalloc vs. hoard vs. jemalloc
#12tcmalloc has a lot of parameters, and the defaults are not suited to any workload I've ever seen, so I'd be interested to know if these knobs were turned for this benchmark. Things like total thread cache size make a huge difference. Also note that the tcmalloc being tested here, from the Debian package, lacks the fast-path improvements released in gperftools 2.6, which represented a roll-up of years of internal Goog…
Good point but adjusting allocators from default is a separate task - which is better to be done by fans of respective allocator. If somebody is able to adjust their-favorite-allocator so results with this-publicly-available-test become better - we'll be happy to publish results with such tuning :-).
Re: Testing Memory Allocators: ptmalloc2 vs. tcmalloc vs. hoard vs. jemalloc
#13Seems like Rust has made a solid choice in making jemalloc their default allocator.
I’d assumed it was the Mozilla legacy - Gecko uses jemalloc, and there’ll be a lot of institutional knowledge around tuning it.
Re: Testing Memory Allocators: ptmalloc2 vs. tcmalloc vs. hoard vs. jemalloc
#14Before a majority of Operating Systems adopted Bonwicks Slab allocator, custom allocators made sense. These days, it's best to let the OS worry about the allocations since it knows about available storage devices (eg. NNVM), cache levels and threads, power management, etc, and it can return overallocated (yet unused) memory which custom allocators cannot.
FWIW: all allocators tested are user-space allocators - and moreover, to achieve this kind of performance they MUST be user-space (as user-kernel switch with all the checks is waaaay too expensive). And as soon as we're user-space - it IS possible to compete with whatever-allocator-is-provided-by-your-user-space-lib (not that it makes sense for 99.9% of the projects, but it MIGHT be possible to write an allocator which is better-than-). In addition, it IS possible to write allocators MUCH-better-suited-for-specific-apps (see, for example, talk on Arena Allocators on CPPCON17 by John Lakos from Bloomberg).
Re: Testing Memory Allocators: ptmalloc2 vs. tcmalloc vs. hoard vs. jemalloc
#15(As background, I'm a jemalloc developer; reposting my twitter comment on the same article): This is quite a bad way of doing a malloc benchmark -- getting realistic activity patterns is critical (see e.g. Wilson et al.'s survey). It doesn't meaningfully test inter-thread interactions, and randomizes in a way that hurts the effectiveness of thread-local caching.
For the large majority of server workloads on Linux, jemalloc or tcmalloc is probably the right choice of allocator. Trying these out (and spending a few additional test runs tuning their configuration) will often yield significant wins compared to the glibc allocator.
Re: Testing Memory Allocators: ptmalloc2 vs. tcmalloc vs. hoard vs. jemalloc
#16Try to do a bitmap allocator. Each word of memory has a free/used bit attached in a separate map. It allows quick resizing before or after the block without moving. It can also be optimal by reducing the fragmentation (find the best block for a given size in the map). It can be optimized with bit counting intrinsics (like popcntl()).
Re: Testing Memory Allocators: ptmalloc2 vs. tcmalloc vs. hoard vs. jemalloc
#17I was trying to implement my own malloc and free, but I couldn't figure out how to validate the correctness of my implementation. Does anybody know of any test suite for malloc/free?
Edit: I should have read TFA.
Re: Testing Memory Allocators: ptmalloc2 vs. tcmalloc vs. hoard vs. jemalloc
#18This benchmark tested glibc 2.24. glibc 2.26 introduced a newer, much more scalable implementation of malloc, with a per-thread cache. I'd love to see how that competes with jemalloc and the other allocators.
Re: Testing Memory Allocators: ptmalloc2 vs. tcmalloc vs. hoard vs. jemalloc
#19I was trying to implement my own malloc and free, but I couldn't figure out how to validate the correctness of my implementation. Does anybody know of any test suite for malloc/free? Edit: I should have read TFA.
LD_PRELOAD?
Re: Testing Memory Allocators: ptmalloc2 vs. tcmalloc vs. hoard vs. jemalloc
#20For more jemalloc information/context/history/trivia, I would recommend reading the jemalloc paper.
"A Scalable Concurrent malloc(3) Implementation for FreeBSD", Jason Evans, April 16, 2006