The micro benchmarks are great, the whole program level benchmarks show nothing earth shaking. But I'll take ok performance and half the code any day.
SuperMalloc: A Super Fast Multithreaded Malloc for 64-bit Machines
11–20 of 33 posts
Re: SuperMalloc: A Super Fast Multithreaded Malloc for 64-bit Machines
#12"dual licensed" - guaranteed to double your licensing confusion.
Can't we just use the MIT part? What's the point of having two FOSS licenses, one being non-viral?
The author is giving you the option to omit the MIT text and have a pure GPLv3 if you wish to restrict access to your software in that way.
Re: SuperMalloc: A Super Fast Multithreaded Malloc for 64-bit Machines
#13Re: SuperMalloc: A Super Fast Multithreaded Malloc for 64-bit Machines
#14Earlier quoted context omitted.
Can't we just use the MIT part? What's the point of having two FOSS licenses, one being non-viral?
I don't think adding a GPLv3 dual to an MIT license changes much. You can derive a work from MIT licensed code and license your work with a more restrictive license. You would still need to reproduce the MIT license alongside your GPLv3 license which would be confusing. The author is giving you the option to omit the MIT text and have a pure GPLv3 if you wish to restrict access to your software in that way.
The MIT license must be included in the provided software, but it isn't a part of the new license.
Re: SuperMalloc: A Super Fast Multithreaded Malloc for 64-bit Machines
#15Earlier quoted context omitted.
Interesting. "In August 2014, Intel announced a bug in the TSX implementation on current steppings of Haswell, Haswell-E, Haswell-EP and early Broadwell CPUs, which resulted in disabling the TSX feature on affected CPUs via a microcode update." -- So only (non-early) Broadwell CPUs support it now?
As well as Skylake CPUs now. Check in the "TSX-NI" table entry. http://ark.intel.com/products/88191/Intel-Core-i5-6600K-Proc...
Re: SuperMalloc: A Super Fast Multithreaded Malloc for 64-bit Machines
#16For what it's worth, in my tests, Hoard ( http://www.hoard.org , https://github.com/emeryberger/Hoard ) significantly outperformed SuperMalloc on a machine without TSX. For example, on a simple microbenchmark that tests scalability, Hoard is 2x faster with one thread, and the gap widens with more threads (Hoard is 3.3x faster with 8 threads, 4x faster with 16 threads).
Your papers were very valuable while doing my undergraduate study. Is Hoard neck and neck with jemalloc and lockless' allocator nowadays?
Re: SuperMalloc: A Super Fast Multithreaded Malloc for 64-bit Machines
#17Earlier quoted context omitted.
Can't we just use the MIT part? What's the point of having two FOSS licenses, one being non-viral?
I don't think adding a GPLv3 dual to an MIT license changes much. You can derive a work from MIT licensed code and license your work with a more restrictive license. You would still need to reproduce the MIT license alongside your GPLv3 license which would be confusing. The author is giving you the option to omit the MIT text and have a pure GPLv3 if you wish to restrict access to your software in that way.
Re: SuperMalloc: A Super Fast Multithreaded Malloc for 64-bit Machines
#18Earlier quoted context omitted.
TSX is disabled in most (all?) Haswell SKUs due to bugs.
Interesting. "In August 2014, Intel announced a bug in the TSX implementation on current steppings of Haswell, Haswell-E, Haswell-EP and early Broadwell CPUs, which resulted in disabling the TSX feature on affected CPUs via a microcode update." -- So only (non-early) Broadwell CPUs support it now?
Re: SuperMalloc: A Super Fast Multithreaded Malloc for 64-bit Machines
#19The author of this paper does not use lock-free techniques, which our allocator used - I'm curious if using lock-free algorithms would have changed the author's design, or improved the performance. I do think that despite the similarities that Streamflow has to TBBmalloc, Streamflow is not susceptible to the same kind of memory blowup. The problem, as described in the recent paper:
"TBBmalloc can have an unbounded footprint. One case was documented by [40]. In this case, one thread allocates a large number of objects, and a second thread then frees them, placing them into the first threads foreign block. If the first thread then does not call free(), then the memory will never be removed from the foreign block to be reused. There appears to be no easy fix to this problem in TBBmalloc, since the thread-local locking policy assumes, deep in its design, that every thread calls free() periodically."
Streamflow avoids this problem by putting the remote-free block check on the malloc path. That is, when allocating memory, you always check if other threads remotely freed memory for you. Basically, if you're continually allocating memory, you're also continually checking to see if you should clean your memory that other threads freed for you. You can see this in action: https://github.com/scotts/streamflow/blob/master/streamflow....
All of the above is just to add some background - I look forward to really digging into this paper over the holidays.
(In case anyone actually follows my comments, they may know I currently do research and development for IBM Streams. That has zero relationship to this memory allocator I worked on early in grad school; it's just an odd coincidence of project names.)