Testing Memory Allocators: ptmalloc2 vs. tcmalloc vs. hoard vs. jemalloc • Hacker News Board

Re: Testing Memory Allocators: ptmalloc2 vs. tcmalloc vs. hoard vs. jemalloc

#41

Post by ksec » Thu, Jul 05, 2018, 7:38 PM UTC

pletnes wrote: post #23

What happens if you replace the glibc malloc used in implementations of higher level languages like python/ruby/js? Aren’t these languages big on allocating small objects very often?

There has been lots of discussion lately about Ruby and malloc. jemalloc gives the best results for Server Performance and memory usage [1], but is slightly slower in low memory usage scenario i.e Non Server usage which is popular in Japan. But jemalloc sometimes ( Not sure if this is still the case ) doesn't work well with muslc, used in Alpine Linux.

The Ruby Core team has decided against [2] shipping jemalloc by default, but will use it when it is available. Which ultimately led to some work on SleepyGC [3] and Transient heap [4].

[1] https://www.mikeperham.com/2018/04/25/taming-rails-memory-bl...

[2] https://bugs.ruby-lang.org/issues/14718

[3] https://bugs.ruby-lang.org/issues/14723

[4] https://bugs.ruby-lang.org/issues/14858

Re: Testing Memory Allocators: ptmalloc2 vs. tcmalloc vs. hoard vs. jemalloc

#42

Post by sanxiyn » Fri, Jul 06, 2018, 2:05 AM UTC

robin_reala wrote: post #13

pcstl wrote: post #11

Seems like Rust has made a solid choice in making jemalloc their default allocator.

I’d assumed it was the Mozilla legacy - Gecko uses jemalloc, and there’ll be a lot of institutional knowledge around tuning it.

This is correct.

Re: Testing Memory Allocators: ptmalloc2 vs. tcmalloc vs. hoard vs. jemalloc

#43

Post by gliptic » Tue, Jul 10, 2018, 9:58 AM UTC

mabynogy wrote: post #28

gliptic wrote: post #24

Earlier quoted context omitted.

1 bit per byte would require an awful lot of memory access to find a free block. You would need something like a hierarchy of bitmaps to cut down on scanning.

No because you'll end up with the same problem elsewhere (handling blocks). If I'm correct tcmalloc memory overhead is around 4%. More overhead can be acceptable if the memory is less fragmented (less fragmentation, less cache misses, better compactness and better perfs).

Well, you're welcome to try, but it's not like free space bitmaps are unknown to allocator writers.