One has to wonder if this due to the global memory shortage. ("Oh - changing our memory allocator to be more efficient will yield $XXM dollar savings over the next year").
Meta’s renewed commitment to jemalloc
11–20 of 259 posts
Re: Meta’s renewed commitment to jemalloc
#12I recently started using Microsoft's mimalloc (via an LD_PRELOAD) to better use huge (1 GB) pages in a memory intensive program. The performance gains are significant (around 20%). It feels rather strange using an open source MS library for performance on my Linux system. There needs to be more competition in the malloc space. Between various huge page sizes and transparent huge pages, there are a lot of gains to be…
If there is so much performance difference among generic allocators, it means you need semantic optimized allocators (unless performance is actually not that much important in the end).
Re: Meta’s renewed commitment to jemalloc
#13I recently started using Microsoft's mimalloc (via an LD_PRELOAD) to better use huge (1 GB) pages in a memory intensive program. The performance gains are significant (around 20%). It feels rather strange using an open source MS library for performance on my Linux system. There needs to be more competition in the malloc space. Between various huge page sizes and transparent huge pages, there are a lot of gains to be…
One of the best parts about GC languages is they tend to have much more efficient allocation/freeing because the cost is much more lumped together so it shows up better in a profile.
Re: Meta’s renewed commitment to jemalloc
#14Meta never abandoned jemalloc. https://github.com/facebook/jemalloc remained public the entire time. It's my understanding that Jason Evans, the creator of jemalloc, had ownership over the jemalloc/jemalloc repo which is why that one stopped being updated after he left.
Re: Meta’s renewed commitment to jemalloc
#15Meta never abandoned jemalloc. https://github.com/facebook/jemalloc remained public the entire time. It's my understanding that Jason Evans, the creator of jemalloc, had ownership over the jemalloc/jemalloc repo which is why that one stopped being updated after he left.
The repo's availability isn't related to whether it's still maintained.
Re: Meta’s renewed commitment to jemalloc
#16Meta never abandoned jemalloc. https://github.com/facebook/jemalloc remained public the entire time. It's my understanding that Jason Evans, the creator of jemalloc, had ownership over the jemalloc/jemalloc repo which is why that one stopped being updated after he left.
The repo's availability isn't related to whether it's still maintained.
Meta still maintained it and actively pushed commits to it fixing bugs and adding improvements. From this blog post it sounds like they are increasing investment into it along with resurrecting the original repo. When the repo was archived Meta said that development on jemalloc would be focused towards Meta's own goals and needs as opposed to the larger ecosystem.
Re: Meta’s renewed commitment to jemalloc
#17Someone should tell Bryan Cantrill, he'd probably be ecstatic...
Re: Meta’s renewed commitment to jemalloc
#18I recently started using Microsoft's mimalloc (via an LD_PRELOAD) to better use huge (1 GB) pages in a memory intensive program. The performance gains are significant (around 20%). It feels rather strange using an open source MS library for performance on my Linux system. There needs to be more competition in the malloc space. Between various huge page sizes and transparent huge pages, there are a lot of gains to be…
If you go into Dr Dobbs, The C/C++ User's Journal and BYTE digital archives, there will be ads of companies whose product was basically special cased memory allocator.
Even toolchains like Turbo Pascal for MS-DOS, had an API to customise the memory allocator.
The one size fits all was never a solution.
Re: Meta’s renewed commitment to jemalloc
#19Is there a concise timelime/history of this? I thought jemalloc was 100% open source, why is Meta in control of it?
Re: Meta’s renewed commitment to jemalloc
#20I recently started using Microsoft's mimalloc (via an LD_PRELOAD) to better use huge (1 GB) pages in a memory intensive program. The performance gains are significant (around 20%). It feels rather strange using an open source MS library for performance on my Linux system. There needs to be more competition in the malloc space. Between various huge page sizes and transparent huge pages, there are a lot of gains to be…
One of the best parts about GC languages is they tend to have much more efficient allocation/freeing because the cost is much more lumped together so it shows up better in a profile.
Agreed, however there is also a reason why the best ones also pack multiple GC algorithms, like in Java and .NET, because one approach doesn't fit all workloads.