Any thoughts about snmalloc as an alternative? Can't quite recall the differences but seemed to look good on benchmarks.
Spotting and avoiding heap fragmentation in Rust applications
21–30 of 70 posts
Re: Spotting and avoiding heap fragmentation in Rust applications
#22"The specific cause for the fragmentation could be any number of things: JSON parsing with serde, something at the framework-level in axum, something deeper in tokio, or even just a quirk of the specific allocator implementation for the given system. Even without knowing the root cause (if there is such a thing) the behavior is observable in our environment and somewhat reproducible in a bare-bones app." So what is t…
The gold standard for avoiding fragmentation is what jemalloc does, that is, only allocating objects of similar size from a chunk of memory. That is, instead of a single global heap there exists a pool for every valid size of object (and to keep the numbers low, object sizes are rounded up to some set of buckets).
This means that there is more memory wasted for small programs, but as memory use grows the wastage caused by this remains constant and allocation and deallocation will always remain fast.
Re: Spotting and avoiding heap fragmentation in Rust applications
#23Shouldn't this also be a much larger problem when deploying rust on embedded systems or in the kernel? How is that solved in these cases? Or is this not a problem at all?
Rust by default uses the platform allocator. Anything that's true of it's allocator performance is also true of C.
Also, as the sibling comment said, when you are doing tight embedded, the solution is not to allocate things dynamically.
Re: Spotting and avoiding heap fragmentation in Rust applications
#24Shouldn't this also be a much larger problem when deploying rust on embedded systems or in the kernel? How is that solved in these cases? Or is this not a problem at all?
The embedded code I write doesn’t use an allocator at all, so it’s not a problem there.
Re: Spotting and avoiding heap fragmentation in Rust applications
#25In some ways Rust is in the worst possible position in terms of language design when it comes to fragmentation.
In a completely manually managed language like C or C++, you can handle fragmentation problems yourself by writing your own allocators or doing object pools to reuse previously allocated memory. You have control over fragmentation.
In a garbage collected language like Java or C#, the runtime is able to move objects around in memory and update their pointers. So, as long as your language has a sufficiently advanced implementation, it may defragment on the fly for you.
But Rust is sort of stuck in the middle. It's safe enough that it's hard to write your own allocators or easily reuse previously allocated memory. But it's low level enough that the runtime doesn't have the freedom to move things around in memory under the program.
It's a hard problem.
Re: Spotting and avoiding heap fragmentation in Rust applications
#26"The specific cause for the fragmentation could be any number of things: JSON parsing with serde, something at the framework-level in axum, something deeper in tokio, or even just a quirk of the specific allocator implementation for the given system. Even without knowing the root cause (if there is such a thing) the behavior is observable in our environment and somewhat reproducible in a bare-bones app." So what is t…
Our assumption, which turned out to be true, is that it's due to the JSON parsing code. Rust (serde) is very efficient with parsing JSON to a predefined structure, but when it comes to parsing to a "generic object", which we need for part of the payload, it's not as much. We are going to deploy a full fix for this issue too, but jemalloc already solved it as well.
Though I disagree with saying it's "just slapping jemalloc on to solve this". The piece of code in question definitely made the fragmentation issue worse, as it was making a lot of allocations of varying sizes, but the underlying issue of memory fragmentation because of the allocator was still there, and it would have just triggered later by a different code path.
Re: Spotting and avoiding heap fragmentation in Rust applications
#27Earlier quoted context omitted.
No - heap fragmentation actually leaves memory unusable by the workload. It may or may not slow down allocation depending on the design of the allocator. (The analogy to file system fragmentation for memory is that when physical pages are allocated in a discontinuous manner, it prevents some optimizations like coalescing them into hugepages, which for some workloads can help with TLB hit rate.)
Filesystem may also run out of inodes, even though there is still plenty of space.
I don't think either FAT32 or NTFS suffer this problem, they allow metadata to become fragmented instead. Certainly a different design philosophy to the unixy file systems.
Re: Spotting and avoiding heap fragmentation in Rust applications
#28> Gaps that are too small and scattered throughout the heap can lead to new "fresh" blocks of memory being allocated to accommodate a new value that won’t fit otherwise. Though unfortunately because of how memory management works a "defrag" is not possible.
This is how memory management works now, but some older systems like classic Mac OS and Palm OS used a design that did make it possible to compact the heap.
See https://en.wikipedia.org/wiki/Classic_Mac_OS_memory_manageme...
When you allocated memory, instead of getting a pointer back, you'd get a handle. While the handle was unlocked, the operating system was free to move the memory around. If you wanted to access the memory, you'd lock the handle, giving a pointer with the actual address, then unlock the handle when done.
To the extent that you kept handles unlocked, the system could fight fragmentation.
It was tedious, and your code had lots of lock/unlock clutter in it. It also led to bugs where you'd accidentally use a pointer value after unlocking its handle, which makes the pointer value invalid because your data might have been moved somewhere else. Worse, usually your data was not moved, so these bugs were hard to detect, much like use-after-free bugs.
But, when memory is very limited and you also don't have an MMU, life isn't easy.
Re: Spotting and avoiding heap fragmentation in Rust applications
#29In some ways Rust is in the worst possible position in terms of language design when it comes to fragmentation. In a completely manually managed language like C or C++, you can handle fragmentation problems yourself by writing your own allocators or doing object pools to reuse previously allocated memory. You have control over fragmentation. In a garbage collected language like Java or C#, the runtime is able to move…
>In a completely manually managed language like C or C++, you can handle fragmentation problems yourself by writing your own allocators or doing object pools to reuse previously allocated memory. You have control over fragmentation.
It's done constantly in Rust. Create a vector of items you want to allocate. Reference to them by their id, i.e. int representing them. There're libraries supporting this style of development, for example, slab: https://crates.io/crates/slab
Re: Spotting and avoiding heap fragmentation in Rust applications
#30Any thoughts about snmalloc as an alternative? Can't quite recall the differences but seemed to look good on benchmarks.
Not OP, but I've tried it in a handful of projects and haven't seen a measurable improvement to performance, memory utilization, or heap fragmentation. That said, it's easy to try out different allocators; I recommend giving it a shot for your specific workload, because IME they tend to perform very differently in different applications.