Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Horror in the Standard Library

zerotier.com

Post reply on HN
Print view View original

201–210 of 222 posts

Re: The Horror in the Standard Library

#201

Post by CamperBob2 »

dom0 wrote: post #185
badsectoracula wrote: post #92

Earlier quoted context omitted.

This is actually a fine thing to do, i don't remember where i read it, but a good analogy for trying to free memory at program exit is like trying to clean the floors and walls of a building right before it is demolished. This is also why Valgrind separates reachable and unreachable memory and only considers unreachable memory as leaks.

A good example is something like doing a "cp -a". To preserve hardlinks you'll need a mapping, trying to free that at exit can and will take time. This was an actual bug in the coreutils.

I have to wonder why the C standard library didn't include a Pascal-style mark/release allocator. A naive implementation wouldn't be much faster than free()'ing a bunch of allocations manually, but the general idea offers possibilities for optimization that otherwise aren't available to a conventional heap allocator.

Re: The Horror in the Standard Library

#202

Post by fnord123 »

CiaranMcNulty wrote: post #58
bostik wrote: post #51

Earlier quoted context omitted.

> you can't trust libraries blindly, even one of the most used and broadly adopted ones There is a corollary to development and debugging. When things break in mysterious ways, we tend to go through a familiar song and dance. As experience, skills and even personal networks grow, we can find ourselves diving ever further in the following chain. 1. "It must be in my code." -- hours of debugging 2. "Okay, it must be so…

11. "Hm maybe I should check my code again... ah there's the bug"

    if (featureFlags[HN_DEBUG_HIER_FLAGS] = null) {
    /* Who won't this trigger!?!?!?
    */
    }
...oh god, kill me now.

Re: The Horror in the Standard Library

#203

Post by nikanj »

kabdib wrote: post #71
bostik wrote: post #63

Earlier quoted context omitted.

Heh. It could be telling that I had to look up the expansion for TLB. CPU cache implementation... holy crap. My ex-coworker has done the vanilla scope thing too and has a 400MHz scope at home. For some reason people like this are not too uncommon in Finnish oldskool[tm] IT scene. I remember how he isolated a latency and concurrency bug to an expensive interrupt handler. Rewriting isolated parts of core kernel code to…

Cache bugs are one of the fun ones. You think you're losing your mind and the people around you would probably agree. A couple of weeks go by, your spouse is ready to fire you, your boss wants to divorce you, and every waking moment is full of race conditions. Four-way stops on the drive to work are a source of stress and you punch buttons in the elevator and worry about firmware bugs. Then you get to your desk and t…

Your writing reminds me of https://www.usenix.org/system/files/1311_05-08_mickens.pdf

Re: The Horror in the Standard Library

#204

Post by dom0 »

CamperBob2 wrote: post #201
dom0 wrote: post #185

Earlier quoted context omitted.

A good example is something like doing a "cp -a". To preserve hardlinks you'll need a mapping, trying to free that at exit can and will take time. This was an actual bug in the coreutils.

I have to wonder why the C standard library didn't include a Pascal-style mark/release allocator. A naive implementation wouldn't be much faster than free()'ing a bunch of allocations manually, but the general idea offers possibilities for optimization that otherwise aren't available to a conventional heap allocator.

Because (m)alloc predates mmap by a decade or so [1], and you expressly don't want stack-like allocation semantics when using malloc (otherwise you'd just put it on a/the stack).

[1] And you cannot have more than one heap without mmap. Without mmap, you only have sbrk = the one heap. On UNIX and those that pretend to be, anyway.

Re: The Horror in the Standard Library

#205

Post by doubleplusgood »

marcosdumay wrote: post #123
doubleplusgood wrote: post #67

Earlier quoted context omitted.

I feel lucky to only have reached item 4; I wouldn't feel confident beyond it, anyway.

I don't think anybody is ever confident beyond level 1. The worst part is those "why the fuck does everything work when I plug a logical analyzer?" moments.

I'm usually well into #3 before I realize I'm in too deep

Re: The Horror in the Standard Library

#206

Post by daemin »

bluejekyll wrote: post #200
kazinator wrote: post #194

Earlier quoted context omitted.

> It turned out that the std::string empty string reference count was just doing a vanilla ++, no locking, nothing, variable not marked volatile, nothing. The whole implementation is smells of silliness, because there is no need to track how many references there are to a global null string, which need not even be dynamically allocated.

I know; I've often wondered if this was changed, never went back to look.

Probably a few years ago, around C++11 time. Where the standard made it not possible to have a Copy-on-Write implementation of string.

Re: The Horror in the Standard Library

#207

Post by daemin »

badsectoracula wrote: post #92
CamperBob2 wrote: post #56

Earlier quoted context omitted.

Notes about deallocation. This allocator does not explicitly release memory. Because of this, memory debugging programs like valgrind or purify may notice leaks: sorry about this inconvenience. Operating systems will reclaim allocated memory at program termination anyway. Wow. This is worth a Linus Torvalds-level rant. Whoever accepted this code into the source tree needs to be put on GNU's version of a performance i…

This is actually a fine thing to do, i don't remember where i read it, but a good analogy for trying to free memory at program exit is like trying to clean the floors and walls of a building right before it is demolished. This is also why Valgrind separates reachable and unreachable memory and only considers unreachable memory as leaks.

This behaviour is one of those cache-memory-leaks. Where even though memory is reachable it is still effectively leaked because it's soaked up by some data structure and not released to the rest of the system.

So it's not a traditional leak but because memory usage would continue to grow it causes the long lived process to choke itself and die.

Re: The Horror in the Standard Library

#208

Post by jwakely »

maxlybbert wrote: post #167
gpderetta wrote: post #162

Earlier quoted context omitted.

>I do think it's silly for operators new/delete to have a separate free list from malloc()/free(). They don't. This is a custom, simple, non-default pool allocator for standard containers (i.e nothing to do with new)

Thanks for the correction. But it's just as silly to create an allocator for the standard containers, when the allocator consists of little more than free list management, given that malloc/free and new/delete already do that. It's especially silly for a library (programs may want custom memory management and libraries really shouldn't go out of their way to make that harder), and the fact that it's the standard libr…

GCC's std::allocator also doesn't do that (not for at least a decade, IIRC). It's a non-default allocator, which nobody is forced to use. It's entirely optional. The default std::allocator just uses new/delete.

Re: The Horror in the Standard Library

#209

Post by jwakely »

grandinj wrote: post #49

I'm guessing the cpp thing is a holdover from the days when the glibc maintainer was less than entirely helpful. There has been actual improvements in glibc in this area lately so hopefully these kinds of hacks will slowly go away.

The pooling behaviour of the libstdc++ std::allocator was only the default from 2004 until late 2005, so it went away more than a decade ago.
Post reply on HN