Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Horror in the Standard Library

zerotier.com

Post reply on HN
Print view View original

91–100 of 222 posts

Re: The Horror in the Standard Library

#91

Post by CJefferson »

jwr wrote: post #79
flamedoge wrote: post #12

Earlier quoted context omitted.

It really bugs me that nobody links bug on bugtracker or filed it. I know I'm asking a lot and being an ass.

No, that's a reasonable expectation. Unfortunately, developers often don't appreciate bug reports and become defensive. File a bug report and you will be expected to provide a reproducible test case, or your bug will get closed. This isn't always possible, and the reporter might not be able to dedicate the time to do this. I used to file lots of bug reports, because I know that as a developer, I'd want them. Every bu…

The problem with libstdc++ is that, in my experience, 90% of bug reports are user error. That means non reproducible bug reports have almost no value.

Re: The Horror in the Standard Library

#92

Post by badsectoracula »

CamperBob2 wrote: post #56
arunc wrote: post #54

I encountered exactly the same issue few years ago in UIDAI in one of our large scale biometric matchers and the resolution was exactly the same. After a week of debugging I found that the libstdc++ allocator was the culprit. I found [1] and confirmed the same, which helped in fixing this issue. The thing that was more interesting (or sad) was to know that the GCC developers didn't expect the multithreaded applicatio…

Notes about deallocation. This allocator does not explicitly release memory. Because of this, memory debugging programs like valgrind or purify may notice leaks: sorry about this inconvenience. Operating systems will reclaim allocated memory at program termination anyway. Wow. This is worth a Linus Torvalds-level rant. Whoever accepted this code into the source tree needs to be put on GNU's version of a performance i…

This is actually a fine thing to do, i don't remember where i read it, but a good analogy for trying to free memory at program exit is like trying to clean the floors and walls of a building right before it is demolished.

This is also why Valgrind separates reachable and unreachable memory and only considers unreachable memory as leaks.

Re: The Horror in the Standard Library

#93

Post by staticlibs »

arunc wrote: post #54

I encountered exactly the same issue few years ago in UIDAI in one of our large scale biometric matchers and the resolution was exactly the same. After a week of debugging I found that the libstdc++ allocator was the culprit. I found [1] and confirmed the same, which helped in fixing this issue. The thing that was more interesting (or sad) was to know that the GCC developers didn't expect the multithreaded applicatio…

Looks like this is just one of possible "non-default" extension allocators, that can be selected during libstdc++ compile time: https://gcc.gnu.org/onlinedocs/libstdc++/manual/memory.html#...

Re: The Horror in the Standard Library

#94

Post by faragon »

The problem is forgetting that dynamic memory usage is not "free" (as in "gratis" or "cheap"). In fact, using std::string for long-lived server processes doing intensive string processing (e.g. parsing, text processing, etc.) is already known to be suicidal since forever, because of memory fragmentation.

For high load backend processing data, you need at least a soft real-time approach: avoid dynamic memory usage at runtime (use dynamic memory just at process start-up or reconfig, and rely on stack allocation for small stuff, when possible).

I wrote a C library with exactly that purpose [1], in order to work with complex data (strings -UTF8, with many string functions for string processing-, vectors, maps, sets, bit sets) on heap or stack memory, with minimum memory fragmentation and suitable for soft/hard real-time requirements.

[1] https://github.com/faragon/libsrt

Re: The Horror in the Standard Library

#95

Post by rhaps0dy »

saurik wrote: post #47
squidlogic wrote: post #22

Amazing write-up. Informative and gripping in its prose.

This article was an extremely long rant about "something has a if but templates are hard so I have no clue what it is". The author figured out a workaround to the issue, but we still don't know what the bug was, and the strongly worded conclusion that it is a bug in libstdc++ isn't even defended well (as this is similar to concluding "there is a bug in the compiler's code optimizer" from "I compiled my code with -O0…

I'm curious, what is usually the cause of code starting to work when it is compiled with -O0?

Re: The Horror in the Standard Library

#97

Post by kyberias »

s_kilk wrote: post #80
bostik wrote: post #51

Earlier quoted context omitted.

> you can't trust libraries blindly, even one of the most used and broadly adopted ones There is a corollary to development and debugging. When things break in mysterious ways, we tend to go through a familiar song and dance. As experience, skills and even personal networks grow, we can find ourselves diving ever further in the following chain. 1. "It must be in my code." -- hours of debugging 2. "Okay, it must be so…

I've only seen level 5 personally, with a C# compiler bug that would omit totally valid `else` branches.

Care to elaborate? Which C# compiler?

Re: The Horror in the Standard Library

#98

Post by s_kilk »

kyberias wrote: post #97
s_kilk wrote: post #80

Earlier quoted context omitted.

I've only seen level 5 personally, with a C# compiler bug that would omit totally valid `else` branches.

Care to elaborate? Which C# compiler?

The official one, I think it was in .NET 3, but it was a few years ago at an old job, so I'm a bit hazy on the details.

Basically we had a bug where a whole conditional branch was being skipped, and we traced it down to the branch being omitted entirely from the compiled IR.

And no, it was nothing fancy, just something like:

    if (customer.country == "US") {
      doSomething();
    } else {
      doDifferentThing();
    }
The whole `else` branch was simply missing from the compiled program.

If I remember correctly, we got around it by doing something like:

    bool isUsCustomer = customer.country == "US"
    if (isUsCustomer) {
      doSomething();
    } else {
      doDifferentThing();
    }
Anyway, the point is that the compiler fucked up it's handling of if/else statements, but only at that specific part of the code, leading to a few wasted days of effort tracking down the problem.

Re: The Horror in the Standard Library

#99

Post by bostik »

rhaps0dy wrote: post #95
saurik wrote: post #47

Earlier quoted context omitted.

This article was an extremely long rant about "something has a if but templates are hard so I have no clue what it is". The author figured out a workaround to the issue, but we still don't know what the bug was, and the strongly worded conclusion that it is a bug in libstdc++ isn't even defended well (as this is similar to concluding "there is a bug in the compiler's code optimizer" from "I compiled my code with -O0…

I'm curious, what is usually the cause of code starting to work when it is compiled with -O0?

I don't have an answer to that, but I have seen - and worked with - several codebases which work fine when compiled with -g, but will crash (in good cases) or behave irrationally (in bad cases) without.

The crashing ones at least are easy. Somewhere a list or variable-argument array is missing the NULL terminator...

Re: The Horror in the Standard Library

#100

Post by RX14 »

bostik wrote: post #51
bluejekyll wrote: post #29

OMG, as I was reading this I thought, "man, this reminds me of a bug I ran into with std::string back in 2000", A few sentences later, and this is also about std::string and the STL. Mine was different though, after tracking down a memory leak that was happening with the creation of just new empty string, I discovered in the stdlib that there was a shared pointer to the empty string with a reference count of how many…

> you can't trust libraries blindly, even one of the most used and broadly adopted ones There is a corollary to development and debugging. When things break in mysterious ways, we tend to go through a familiar song and dance. As experience, skills and even personal networks grow, we can find ourselves diving ever further in the following chain. 1. "It must be in my code." -- hours of debugging 2. "Okay, it must be so…

This is the worst thing I've ever found, still not solved: https://github.com/crystal-lang/crystal/issues/4127
Post reply on HN