Looks like the same idea as Konstantin Knizhnik's thread_alloc:
http://www.garret.ru/threadalloc/readme.html
At least the same architecture of allocated chunks management.
Mimalloc – A compact general-purpose allocator
21–30 of 70 posts
Re: Mimalloc – A compact general-purpose allocator
#22they should not have tested on AWS
Re: Mimalloc – A compact general-purpose allocator
#23The tricky part with allocators is always the multi-threaded setups. Even something as simple as a bunch of threads doing malloc-free in a loop will drop performance of a lot of allocators to the floor, due to some sort of central locking or excessive cache thrashing. This is typically solved by adding per-thread block pools, free lists or some such. If you go further down the rabbit hole, there's a case when blocks…
You should actually read the technical report: https://www.microsoft.com/en-us/research/uploads/prod/2019/0...
This has some quite clever and very carefully considered details around concurrency.
> in the end this is a very well-explored area and there's basically a single stable point once all common scenarios are considered.
I think it's absurd to call allocation a solved problem where there's one known optimal design.
Re: Mimalloc – A compact general-purpose allocator
#24I never like names that require a “pronounced like” note, but cool project regardless.
Re: Mimalloc – A compact general-purpose allocator
#25We tried mimalloc in ClickHouse and it is two times slower than jemalloc in our common use case
https://github.com/microsoft/mimalloc/issues/11
Re: Mimalloc – A compact general-purpose allocator
#26I never like names that require a “pronounced like” note, but cool project regardless.
Given the rather unpredictable nature of English pronunciation that's basically required for any made up word.
Re: Mimalloc – A compact general-purpose allocator
#27Are there functions available with which I can at run-time query how much OS memory is used, how much handed out in allocations, how many mmap()ed pools are used, and so on?
I find that one of the most important features of a malloc library to debug memory usage.
glibc has these functions (like malloc_info()) -- they are very bugged in that they return wrong results, but after patching them to be correct, they are super useful.
Re: Mimalloc – A compact general-purpose allocator
#28The tricky part with allocators is always the multi-threaded setups. Even something as simple as a bunch of threads doing malloc-free in a loop will drop performance of a lot of allocators to the floor, due to some sort of central locking or excessive cache thrashing. This is typically solved by adding per-thread block pools, free lists or some such. If you go further down the rabbit hole, there's a case when blocks…
but in the end this is a very well-explored area
With a lot of interesting new generic allocators that beat the competition hands down popping up in the last 10 years, I'd dare to claim the area is not very well explored. It has indeed been explored a lot but it seems that there's still low-hanging fruit. Every few years someone writes an allocator that makes significant improvements over the previous generation implementations. I love to see such action on "solved problems".
Re: Mimalloc – A compact general-purpose allocator
#29We tried mimalloc in ClickHouse and it is two times slower than jemalloc in our common use case https://github.com/microsoft/mimalloc/issues/11
To be clear, your program ran at half speed, right? That's far worse than doubling the time spent in memory-management functions.
Re: Mimalloc – A compact general-purpose allocator
#30We tried mimalloc in ClickHouse and it is two times slower than jemalloc in our common use case https://github.com/microsoft/mimalloc/issues/11
[deleted]