The effect of switching to TCMalloc on RocksDB memory use

For my bigger libraries glibc malloc and free has insane runtimes. Eg a final free could last over 1 minute. If you see this with a GC you would throw it out immediately. But apparently people don't measure and spread false rumors about GC's. Even a very poor and primitive GC as in emacs is not that slow as glibc malloc/free. A good one is miles faster. My GC has an upper limit of 10ms, not minutes.

clearly there are circumstances where this is not an option, but for distributed systems that can handle the short-term loss of a given node it works very well. one important thing is to add is some randomized jitter to the periodic restarts to spread out when nodes are offline.

in practice this remedies a wide range of seldom-encountered problems. for instance, I have a program that pulls data from an http apis, and last week its connection to one of the remote servers became hung up somehow, resulting in no new data being pulled from that server. this problem had never happened before in 6 months since the program was deployed. rather than track down the very rare condition that was behind this I just added periodic "partial" restarts to the program, and now it will be able to recover from this if it ever happened again, and many other sorts of problems like slowing increasing memory fragmentation.

for a chat server or other server with client connections, obviously those would need to be maintained across the partial restart, but that seems like it would be pretty easy to do. while there is nothing dramatically different from my approach to what you did with actual, full periodic restarts, the partial restart approach does allow special handling for application-specific issues like that.

Indeed. And is Cheesecake a pie or a cake? You bake cheesecake in pie-crust, but it has cake in the name.

The memory-management "its both" scheme is reference-counting. In some contexts, its ref-counts are considered manual (C++ shared_ptr, Rust). In other contexts, its considered automated garbage collection (Python, Lisp).

In all cases: tons of atomic add/subtract + memory-barriers to strictly order reference counts between many threads... making it actually kind of bad for multithreaded code compared to other garbage collection schemes. (Each shared_ptr = blah() requires an atomic add and atomic-subtract. So you actually have escalating costs the more you use the shared_ptr). I think the circular-issue is kind of overblown: it certainly happens (doubly-linked lists, graphs, trees which store a "root pointer" or even an "iterator" somewhere inside of them) but people know how to use weak_ptr these days and avoid most issues. (Well... except for arbitrary graphs. No way to know where to put weak_ptr there. But most "graph heavy" algorithms use other representations: dense matrix or sparse matrix forms... and probably should avoid slow / low-performance general purpose memory allocators anyway)

-----------

Once you recognize the multithreaded-sync issues associated with shared_ptr / RefCount, you start building a thread that "centralizes" the ref-counts in a reader/writer queue to ensure that objects are deleted at the right time. Oh wait, that's called a garbage collector thread and you've suddenly moved to mark&sweep accidentally.

Once you get a garbage collector thread, its not too big of a leap to start thinking about multiple garbage-collector threads (scaling to higher garbage collection performance). I mean, yeah, I'm glossing over all sorts of high-performance multithreaded lock-free datastructures here, but lets pretend those implementation details are solved. Lol.

-----------

As usual: its not really the garbage collection or memory-management parts that are hard. Its the multithreaded high-performance (and provably correct!) portion that's really, really hard. Even for simple ideas like ref-counts or even "manual" malloc/free.

The 4.0 version of cassandra is finally doing the smart thing and spinning up separate threads dedicated to sets/shards of hash ranges held by a node. I'm not sure if they'll run different JVMs per thread as well to do further sharding of the GC generations as well or if they can effectively do that from one JVM, but that would make sense