The "make malloc faster" part was done over a decade ago with the followup from ptmalloc2 (the official glibc malloc) to ptmalloc3. But it added one word overhead per region, so the libc people never updated it to v3. perf. regression.
They rather broke the workarounds they added. And now they are breaking emacs with their new malloc.
The Horror in the Standard Library
41–50 of 222 posts
Re: The Horror in the Standard Library
#42Things like this is why I was happy to see the LLVM project write their own C++ standard library. libstdc++ has always seemed a bit hacky and fragile to me. It's great to have an option which is a more modern, clean codebase. Have you tested to see if this works better with LLVM libc++?
Alternatively, things like this is why it is great when everyone works together to improve one lib rather instead of forming their efforts on two essentially-identical ones. You act like
libc++ is somehow simply better and probably doesn't have bugs :/. I have been doing C++ work now for over two decades and let me set that record straight in your head: incredibly basic stuff has been pretty extremely broken in libc++. One horror story that wasted way way too much of my life is that the copy that Mac OS X 10.7 seriously shipped with a build of libc++ where std::streambuf failed to check EOF conditions correctly. Despite most of my projects being compiled simultaneously by numerous versions of both gcc and clang (to target various weird configurations), I seriously don't remember the last time I ran into a bug in gcc and libstdc++... it was pre-2006 for sure... but I continue to run into annoying issues with clang and libc++. The correct way to read "modern" when applied to "codebase" is "untested". And hell: while I am totally willing to believe there is a bug here, this post doesn't have a fix and doesn't even seem to have led to a bug report. This is like saying "I compiled my code using -O0 and it started working, so clearly this is a bug in the optimizer", which we should all know is a dubious statement at best.
Re: The Horror in the Standard Library
#43Yeah malloc() is pretty terrible in glibc by modern standards. For some workloads it just can't keep up and ends up fragmenting space in such a way that memory can't be returned to the OS (and thus be used for the page cache) and you end up in this performance spiral. I always deploy C++ server on jemalloc. Been doing it for years and while there's been occasional hicks up when updating it has provided much more pred…
This has nothing to do with glibc malloc()
It has. It was never updated to ptmalloc3
Re: The Horror in the Standard Library
#44Things like this is why I was happy to see the LLVM project write their own C++ standard library. libstdc++ has always seemed a bit hacky and fragile to me. It's great to have an option which is a more modern, clean codebase. Have you tested to see if this works better with LLVM libc++?
[deleted]
Re: The Horror in the Standard Library
#45Actually it is C's malloc and free that is "broken". malloc() takes a size parameter, but free() doesn't. This imbalance means it can never be maximally efficient. Whatever GNU stdlibc++ is doing is probably, on balance, a net win for most programs. It's not exactly roses in C++ either of course. You can do better than the standard library facilities. Andrei Alexandrescu gave a great, entertaining, and technically el…
If free() took a size, what would that buy you? You are aware that malloc implementations tend to stick the size just before the part returned to the caller, right? eg. let's say you store size at p, return p+4 to caller, then have free() subtract 4 again to get at this "header"... So I'm guessing that's not your suggestion because free() wouldn't work at all without that kind of hack. Or more broadly, free() or real…
They only store the size in a header because free() needs to know the size on deallocation. Think about all the C string functions that take buffer length parameters and what an outcry there would be if we eliminated all the size parameters and placed string lengths as headers before the buffer itself.
It's terrible for performance because you're giving up control, even if it is a little safer overall.
Re: The Horror in the Standard Library
#46I myself ran across this same scenario many years ago with a similar amount of hair pulling and eventually concluding that the GNU libstdc++ allocator wasn't reusing memory properly. Unfortunately, I was never able to pare down the application to the point that I had a reproducible test case to report upstream. GLIBCPP_FORCE_NEW was the solution for the near term and since I was deploying on Solaris boxes I eventuall…
It really bugs me that nobody links bug on bugtracker or filed it. I know I'm asking a lot and being an ass.
Yeah: people run into issues and somehow feel "obviously everyone else ran into this same issue and it was never fixed so they must not want to fix it" without it ever really occurring to them that maybe almost no no one runs into this problem and no one knows it exists, or even that it could be their code that is broken. We all know "I compiled my code with -O0 and it started working" is almost never due to "there is a bug in the code optimizer", and this one sounds extremely similar. If people think they have found a bug, and they want to feel righteous and haughty over how it never got fixed, they really need to be demonstrating it is actually a known bug (and maybe you were being sarcastic, but I will just state flatly: being bothered when people do not does not make you the ass).
Re: The Horror in the Standard Library
#47Amazing write-up. Informative and gripping in its prose.
This article was an extremely long rant about "something has a if but templates are hard so I have no clue what it is". The author figured out a workaround to the issue, but we still don't know what the bug was, and the strongly worded conclusion that it is a bug in libstdc++ isn't even defended well (as this is similar to concluding "there is a bug in the compiler's code optimizer" from "I compiled my code with -O0 and it started working")... I can't really see calling this "informative" :(.
Re: The Horror in the Standard Library
#48I don't understand the point of this article... if you think there's a bug in the library, fix it. Don't write a melodramatic blog post lamenting how horrible it is in the hope somebody else will do it for you.
This isn't particle physics, it's code: we don't have to guess, we can look at it and see how it works.
Re: The Horror in the Standard Library
#49I'm guessing the cpp thing is a holdover from the days when the glibc maintainer was less than entirely helpful. There has been actual improvements in glibc in this area lately so hopefully these kinds of hacks will slowly go away.
Re: The Horror in the Standard Library
#50Post doesn't actually say what was broken, or indeed prove the location of broken-ness. Just that it went away with a different compile option.
Exciting writing, but lacking a point.