Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The effect of switching to TCMalloc on RocksDB memory use

blog.cloudflare.com

Post reply on HN
Print view View original

11–20 of 49 posts

Re: The effect of switching to TCMalloc on RocksDB memory use

#11

Post by flip89 »

KAdot wrote: post #3

It would be interesting to see how jemalloc performs compared to tcmalloc. The article doesn't explain why they choose tcmalloc among other alternative memory allocators.

I've mentioned at the beginning of this article, that we already have several systems running with tcmalloc. So using jemalloc would mean that we would need to bring new dependency. Tcmalloc solves fragmentation issue really well for us, we can get estimation of how much memory is 'lost' due to fragmentation. We saw that potential saving from let's say jemalloc, cannot be higher than 1-3% of memory use and decided it's not worth to bring new dependency for so little value.

Re: The effect of switching to TCMalloc on RocksDB memory use

#12

Post by swiftcoder »

coldcode wrote: post #10

I wrote a commercial memory allocator (long long gone) for pre MacOS 10 and tested it by writing a ton of memory bashing applications based on every perverse pattern I could think of in order to ensure it was fast, failure free and limited fragmentation. Of course from that era I didn't need to support multiple threads so that made it somewhat easier, but in building it (and studying others at the time) it was obviou…

> Computers may be ridiculously fast and have enormous memory today and maybe you can live with the default

Often these hardware improvements exacerbate problems with memory management. For years the runtime of the default JVM garbage collector would blow up past 48 GB of allocations. Which was probably fine for 99% of software out there, but the day your honking great server blew past that... was a bad day all round.

It pays to know your allocators and garbage collectors inside and out for modern cloud scale computing.

Re: The effect of switching to TCMalloc on RocksDB memory use

#13

Post by api »

dragontamer wrote: post #8
api wrote: post #6

Why don’t things like malloc get fixed? I’ve known about these issues for years. Does the core team not know?

Because there's no such thing as a truly general purpose memory allocator. Memory allocation is built to some purpose, and the programmer who uses it should be aware of its problems. There's probably a technical reason why the original glibc malloc does what it does (probably better optimized for low-thread counts)

What you say is technically true but can be used to just hand wave away any issue. Things can almost always be improved.

Allocators like TCMalloc are not significantly worse in single threaded mode. The glibc allocator just isn’t very good.

I have personally fought with the glibc allocator and seen issues with it that don’t exist on other OSes.

Re: The effect of switching to TCMalloc on RocksDB memory use

#14

Post by nateberkopec »

This problem hit the Ruby community hard about 5 years ago.

Basically everyone uses jemalloc or tcmalloc in production in the Ruby community nowadays. What frustrates me is the malloc community's sort of blase reaction along the lines of "there's no bug, this is expected behavior".

Re: The effect of switching to TCMalloc on RocksDB memory use

#15

Post by nerdponx »

An important part of glibc malloc design is that it expects developers to free memory in a reverse order of allocation, otherwise a lot of memory will be ‘locked’, and never returned to the system.

I'm a C novice and generally inexperienced with languages that require memory management. Nobody ever tells you this stuff!

Is this a very specific GNU concern, or do I have to keep this in mind in general on other systems when using their default allocators?

Re: The effect of switching to TCMalloc on RocksDB memory use

#16

Post by vouwfietsman »

tyingq wrote: post #7
KAdot wrote: post #3

It would be interesting to see how jemalloc performs compared to tcmalloc. The article doesn't explain why they choose tcmalloc among other alternative memory allocators.

Microsoft's mimalloc looks interesting also: https://github.com/microsoft/mimalloc HN discussion from 2019: https://news.ycombinator.com/item?id=20249743

For sure mimalloc is interesting!

At work we've been able to use mimalloc to great effect, using it as a drop-in replacement for the standard msvc allocator showed 3x runtime (!) improvements on multi-minute workloads. In terms of memory use we also saw improvements, but less dramatic.

Of course these numbers mean nothing without context (and there was much more heap allocation going on than needed), but getting that kind of performance improvement for (comparatively) such little effort really changed my view on memory and optimization.

Not the most user-friendly library though, perhaps that is a result of its selling point: 0 code changes needed.

Re: The effect of switching to TCMalloc on RocksDB memory use

#17

Post by the_duke »

The blog post sadly doesn't cover what alternatives are out there, and why they settled on their choice.

Besides tcmalloc, there are jmalloc, hoard, the more recent mimalloc, a bunch of commercial options, ...

Would have been interesting to see a comparison.

Edit: background is in this thread: https://news.ycombinator.com/item?id=26013230

Re: The effect of switching to TCMalloc on RocksDB memory use

#18

Post by trasz »

api wrote: post #13
dragontamer wrote: post #8

Earlier quoted context omitted.

Because there's no such thing as a truly general purpose memory allocator. Memory allocation is built to some purpose, and the programmer who uses it should be aware of its problems. There's probably a technical reason why the original glibc malloc does what it does (probably better optimized for low-thread counts)

What you say is technically true but can be used to just hand wave away any issue. Things can almost always be improved. Allocators like TCMalloc are not significantly worse in single threaded mode. The glibc allocator just isn’t very good. I have personally fought with the glibc allocator and seen issues with it that don’t exist on other OSes.

In FreeBSD, the malloc libc uses is jemalloc. Works fine for a general-purpose allocator.

Re: The effect of switching to TCMalloc on RocksDB memory use

#19

Post by curlypaul924 »

the_duke wrote: post #17

The blog post sadly doesn't cover what alternatives are out there, and why they settled on their choice. Besides tcmalloc, there are jmalloc, hoard, the more recent mimalloc, a bunch of commercial options, ... Would have been interesting to see a comparison. Edit: background is in this thread: https://news.ycombinator.com/item?id=26013230

Do you think any of these stand a chance at replacing Doug Lea's malloc in glibc, or does dlmalloc provide more desirable behavior for the general case?

Re: The effect of switching to TCMalloc on RocksDB memory use

#20

Post by flip89 »

nerdponx wrote: post #15

An important part of glibc malloc design is that it expects developers to free memory in a reverse order of allocation, otherwise a lot of memory will be ‘locked’, and never returned to the system. I'm a C novice and generally inexperienced with languages that require memory management. Nobody ever tells you this stuff! Is this a very specific GNU concern, or do I have to keep this in mind in general on other systems…

I think, this particular part of the design is here for historical reasons. The classic way of allocating memory in Unix systems was heap extension with brk/sbrk calls, so glibc malloc from beginning was designed using these syscalls and logic. So with this mechanism you need to free last allocated object to shrink the heap. Now, allocators often get new memory from system using mmap syscall and probably have different expectations.
Post reply on HN