The effect of switching to TCMalloc on RocksDB memory use
blog.cloudflare.com
The effect of switching to TCMalloc on RocksDB memory use
1–10 of 49 posts
Re: The effect of switching to TCMalloc on RocksDB memory use
#2This is really a thing which happens with the glibc allocator and long-lived applications. Back in the day I had a custom chat server which consistently used more and more memory until after about a month it ran out of memory and crashed. I spent quite a while investigating, even annotating every malloc/free in the program to find the leak, only to conclude that there was simply no leak at all in the program and it was some effect in the allocator itself. (Rather than spending time trying other allocators, the "solution" was to schedule a server restart every couple of weeks, early in the morning when no one was using the system.)
Re: The effect of switching to TCMalloc on RocksDB memory use
#3It would be interesting to see how jemalloc performs compared to tcmalloc. The article doesn't explain why they choose tcmalloc among other alternative memory allocators.
Re: The effect of switching to TCMalloc on RocksDB memory use
#4This is really a thing which happens with the glibc allocator and long-lived applications. Back in the day I had a custom chat server which consistently used more and more memory until after about a month it ran out of memory and crashed. I spent quite a while investigating, even annotating every malloc/free in the program to find the leak, only to conclude that there was simply no leak at all in the program and it w…
This is very common in long-lived Ruby applications as well, where jemalloc seems to be the accepted solution.
Re: The effect of switching to TCMalloc on RocksDB memory use
#5I've spent some time really studying memory allocation. It seems to me that almost any high-performance application will take advantage of the fine-details of memory allocators to optimize themselves.
There's something to be said about garbage collection as an actual strategy (!!). I know people think that garbage collection is slow, but... very smart people have put their minds on the memory problem.
A generational garbage collector is designed to tackle fragmentation head-on. Every "generation", the garbage collector recompacts all the data, "squeezing" your fragmentation out of the memory area.
Because some data is "longer lived" than other data, the compacted data is placed into a new generational-pool, where it is left alone for the next collection. IE: All data that survived a collection at time T will be left alone at collect time T+1. Maybe at time T+2 will it be re-checked... and those that survive the check at T+2 will be left alone until T+6.
--------------
This generational strategy minimizes the copying / moving of data around the fragmentation issue. Still though: moving / copying data is the singular catch-all solution to fragmentation.
Re: The effect of switching to TCMalloc on RocksDB memory use
#6Why don’t things like malloc get fixed? I’ve known about these issues for years. Does the core team not know?
Re: The effect of switching to TCMalloc on RocksDB memory use
#7It would be interesting to see how jemalloc performs compared to tcmalloc. The article doesn't explain why they choose tcmalloc among other alternative memory allocators.
Microsoft's mimalloc looks interesting also: https://github.com/microsoft/mimalloc
HN discussion from 2019: https://news.ycombinator.com/item?id=20249743
Re: The effect of switching to TCMalloc on RocksDB memory use
#8Why don’t things like malloc get fixed? I’ve known about these issues for years. Does the core team not know?
Because there's no such thing as a truly general purpose memory allocator.
Memory allocation is built to some purpose, and the programmer who uses it should be aware of its problems. There's probably a technical reason why the original glibc malloc does what it does (probably better optimized for low-thread counts)
Re: The effect of switching to TCMalloc on RocksDB memory use
#9This is really a thing which happens with the glibc allocator and long-lived applications. Back in the day I had a custom chat server which consistently used more and more memory until after about a month it ran out of memory and crashed. I spent quite a while investigating, even annotating every malloc/free in the program to find the leak, only to conclude that there was simply no leak at all in the program and it w…
This is very common in long-lived Ruby applications as well, where jemalloc seems to be the accepted solution.
I've seen this in python as well. gunicorn has two helpful flags to do this automatically (--max-requests and --max-requests-jitter) which will reboot every X number of web requests handled.
Re: The effect of switching to TCMalloc on RocksDB memory use
#10I wrote a commercial memory allocator (long long gone) for pre MacOS 10 and tested it by writing a ton of memory bashing applications based on every perverse pattern I could think of in order to ensure it was fast, failure free and limited fragmentation. Of course from that era I didn't need to support multiple threads so that made it somewhat easier, but in building it (and studying others at the time) it was obvious that this is far from a solved science, and there were terrible design decisions in many common allocators at the time that made little sense.
It clear that some of these decisions still hang around today and that switching to a better allocator really can make a huge difference, especially given everything is multiple threads/cores today. But a lot of programmers never even look at memory usage or investigate alternatives. Even the JVM has a boatload of options on allocators for different usages; yet most Java programmers I know just go with the defaults.
Computers may be ridiculously fast and have enormous memory today and maybe you can live with the default; but it doesn't hurt to look, and might even save you money (pay less to AWS!) by reducing your need to allocate more or bigger servers.