Jemalloc Postmortem
jasone.github.io
Jemalloc Postmortem
1–10 of 250 posts
Re: Jemalloc Postmortem
#2Congrats on the great run and the future. Jemalloc was an inspirational to many memory allocators.
Re: Jemalloc Postmortem
#3Oh that's interesting. jemalloc is the memory allocator used by redis, among other projects. Wonder what the performance impact will be if they have to change allocators.
Re: Jemalloc Postmortem
#4The article mentioned the influence of large-scale profiling on both jemalloc and tcmalloc, but doesn't mention mimalloc. I consider mimalloc to be on par with these others, and now I am wondering whether Microsoft also used large scale profiling to develop theirs, or if they just did it by dead reckoning.
Re: Jemalloc Postmortem
#5Oh that's interesting. jemalloc is the memory allocator used by redis, among other projects. Wonder what the performance impact will be if they have to change allocators.
Why would they have to change? Sometimes software development is largely "done" and there isn't much more you need to do to a library.
Re: Jemalloc Postmortem
#6[deleted]
Re: Jemalloc Postmortem
#7Oh that's interesting. jemalloc is the memory allocator used by redis, among other projects. Wonder what the performance impact will be if they have to change allocators.
Why would they have to change? Sometimes software development is largely "done" and there isn't much more you need to do to a library.
For an example of why an allocator is a maintenance treadmill, consider that C++ recently (relatively) added sized delete, and Linux recently gained transparent huge pages.
Re: Jemalloc Postmortem
#8Oh that's interesting. jemalloc is the memory allocator used by redis, among other projects. Wonder what the performance impact will be if they have to change allocators.
Why would they have to change? Sometimes software development is largely "done" and there isn't much more you need to do to a library.
Technology marches on, and in some number of years other allocators will exist that outperform/outfeature jemalloc.
Re: Jemalloc Postmortem
#9Earlier quoted context omitted.
Why would they have to change? Sometimes software development is largely "done" and there isn't much more you need to do to a library.
Technology marches on, and in some number of years other allocators will exist that outperform/outfeature jemalloc.
This number of years depending on your allocation profile could be something like -10 years easily. New allocators constantly crop up
Re: Jemalloc Postmortem
#10Oh that's interesting. jemalloc is the memory allocator used by redis, among other projects. Wonder what the performance impact will be if they have to change allocators.
Why would they have to change? Sometimes software development is largely "done" and there isn't much more you need to do to a library.
Memory allocators are something I expect to rapidly degrade in the absence of continuous updates as the world changes underneath you. Changing page sizes, new ucode latencies, new security features etc. all introduce either outright breakage or at least changing the optimum allocation strategy and making your old profiling obsolete. Not to mention the article already pointed out one instance where a software stack (KDE, in that case) used allocation profiles that broke an earlier version completely. Even though that's fixed now, any language runtime update or new feature could introduce a new allocation style that grinds you down.
As much as it's nice to think software can be done, I think something so closely tied to the kernel and hardware and the application layer, which all change constantly, never can be.