Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Horror in the Standard Library

zerotier.com

Post reply on HN
Print view View original

111–120 of 222 posts

Re: The Horror in the Standard Library

#111

Post by gone35 »

bostik wrote: post #51
bluejekyll wrote: post #29

OMG, as I was reading this I thought, "man, this reminds me of a bug I ran into with std::string back in 2000", A few sentences later, and this is also about std::string and the STL. Mine was different though, after tracking down a memory leak that was happening with the creation of just new empty string, I discovered in the stdlib that there was a shared pointer to the empty string with a reference count of how many…

> you can't trust libraries blindly, even one of the most used and broadly adopted ones There is a corollary to development and debugging. When things break in mysterious ways, we tend to go through a familiar song and dance. As experience, skills and even personal networks grow, we can find ourselves diving ever further in the following chain. 1. "It must be in my code." -- hours of debugging 2. "Okay, it must be so…

11. "Do we have the IP core for this?"

12. "Where is my electron microscope?"

Re: The Horror in the Standard Library

#112

Post by CJefferson »

fish2000 wrote: post #107
CJefferson wrote: post #91

Earlier quoted context omitted.

The problem with libstdc++ is that, in my experience, 90% of bug reports are user error. That means non reproducible bug reports have almost no value.

I know you were probably just throwing out the “90%” statistic, but if you take that as a given, the implication that 10% of the libstdc++ bugs on file are legit is a worrisome notion in and of its own right. I don’t want to be responsible for triaging those bugs (and nor do you, I am guessing; this being why the reports are valueless) but the fact that this is the bug rate in this, a gold-standard library in common,…

You are right, it's probably a lot less than that.

The bug quality is higher than you might expect, because you have to register with bugzilla, and most bug reports are with development versions, as bugs are shaken out of new features. There are very few bugs in released versions, and where those bugs exist they are often of the form "stupid type where I have redefined & doesn't compile, while the standard technically days it should", where most users would never got them. Wrong answer or crash bugs in releases are extremely rare, although they could be rarer -- the test suite has less coverage than I would personally like.

Re: The Horror in the Standard Library

#113

Post by maxlybbert »

badsectoracula wrote: post #92
CamperBob2 wrote: post #56

Earlier quoted context omitted.

Notes about deallocation. This allocator does not explicitly release memory. Because of this, memory debugging programs like valgrind or purify may notice leaks: sorry about this inconvenience. Operating systems will reclaim allocated memory at program termination anyway. Wow. This is worth a Linus Torvalds-level rant. Whoever accepted this code into the source tree needs to be put on GNU's version of a performance i…

This is actually a fine thing to do, i don't remember where i read it, but a good analogy for trying to free memory at program exit is like trying to clean the floors and walls of a building right before it is demolished. This is also why Valgrind separates reachable and unreachable memory and only considers unreachable memory as leaks.

Raymond Chen often uses this analogy on the blog Old New Thing.

Re: The Horror in the Standard Library

#114

Post by maxlybbert »

CamperBob2 wrote: post #56
arunc wrote: post #54

I encountered exactly the same issue few years ago in UIDAI in one of our large scale biometric matchers and the resolution was exactly the same. After a week of debugging I found that the libstdc++ allocator was the culprit. I found [1] and confirmed the same, which helped in fixing this issue. The thing that was more interesting (or sad) was to know that the GCC developers didn't expect the multithreaded applicatio…

Notes about deallocation. This allocator does not explicitly release memory. Because of this, memory debugging programs like valgrind or purify may notice leaks: sorry about this inconvenience. Operating systems will reclaim allocated memory at program termination anyway. Wow. This is worth a Linus Torvalds-level rant. Whoever accepted this code into the source tree needs to be put on GNU's version of a performance i…

Years ago I read in the Perl documentation ( http://perldoc.perl.org/perlfaq3.html#How-can-I-free-an-arra... ):

> On most operating systems, memory allocated to a program can never be returned to the system. ... Some operating systems (notably, systems that use mmap(2) for allocating large chunks of memory) can reclaim memory that is no longer used ...

Because you can't return unneeded memory to most operating systems (or because it used to be that you couldn't return unneeded memory, even if that has changed recently), it isn't a surprise that by default GCC's free() and operator delete -- which are meant to be cross platform -- don't try to return that memory. Instead it's all free list management.

I do think it's silly for operators new/delete to have a separate free list from malloc()/free().

Re: The Horror in the Standard Library

#115

Post by arglwarg »

bostik wrote: post #51
bluejekyll wrote: post #29

OMG, as I was reading this I thought, "man, this reminds me of a bug I ran into with std::string back in 2000", A few sentences later, and this is also about std::string and the STL. Mine was different though, after tracking down a memory leak that was happening with the creation of just new empty string, I discovered in the stdlib that there was a shared pointer to the empty string with a reference count of how many…

> you can't trust libraries blindly, even one of the most used and broadly adopted ones There is a corollary to development and debugging. When things break in mysterious ways, we tend to go through a familiar song and dance. As experience, skills and even personal networks grow, we can find ourselves diving ever further in the following chain. 1. "It must be in my code." -- hours of debugging 2. "Okay, it must be so…

This one goes to eleven: 11. "On some setups, clients get a corrupted stack when swapped back in from kernel."

(https://bugs.launchpad.net/ubuntu/+source/linux/+bug/745836)

Re: The Horror in the Standard Library

#116

Post by uep »

bostik wrote: post #51
bluejekyll wrote: post #29

OMG, as I was reading this I thought, "man, this reminds me of a bug I ran into with std::string back in 2000", A few sentences later, and this is also about std::string and the STL. Mine was different though, after tracking down a memory leak that was happening with the creation of just new empty string, I discovered in the stdlib that there was a shared pointer to the empty string with a reference count of how many…

> you can't trust libraries blindly, even one of the most used and broadly adopted ones There is a corollary to development and debugging. When things break in mysterious ways, we tend to go through a familiar song and dance. As experience, skills and even personal networks grow, we can find ourselves diving ever further in the following chain. 1. "It must be in my code." -- hours of debugging 2. "Okay, it must be so…

From my experience this is normal for embedded development; particularly for consumer electronics. Part of the reason some developers in this space have to wear so many hats is that the pace in consumer electronics is unforgiving. I don't think my current employer is unusual either.

Hopefully, the number of frameworks at the top, and the size of your individual programs are relatively small (so that 1-3 aren't nightmares by themselves).

In my experience, 4-5 are seldomly the problem (thanks Linaro!). I suspect the ratio of C to C++ is significantly larger in embedded systems though.

In general, PowerPC/MIPS/ARM toolchains and drivers are not as mature as x86/AMD64. 6-8 tend to occur because CPU vendors usually have their own "blessed" toolchains and BSPs that have diverged from their upstream projects. Fortunately, this means that it's often the case that someone else has already fixed the problem. It's just as often that a driver has not been tested for your use-case since the last time that particular driver's infrastructure was refactored inside the kernel. Or... you wrote the driver and made the mistake (or it might be something from 9/10).

9-10 happen because we're often using hardware that is new and has not had all of its errata discovered yet.

When products need to ship, we're regularly going through this stack. I've seen every one of these, even in just the last 4 years.

Re: The Horror in the Standard Library

#117

Post by TimJYoung »

I'm not sure if the other debug tools mentioned offer this, but AQTime Pro:

https://smartbear.com/product/aqtime-pro/overview/

has an allocation profiler that can be used to track down this sort of problem. You can take allocation snapshots while the application is running to see where the allocations are coming from (provided that you can run AQTime Pro against a binary with debug symbols/info).

I'm not affiliated with the company - just a happy customer that has used them for years with Delphi development.

Re: The Horror in the Standard Library

#118

Post by TimJYoung »

bostik wrote: post #51
bluejekyll wrote: post #29

OMG, as I was reading this I thought, "man, this reminds me of a bug I ran into with std::string back in 2000", A few sentences later, and this is also about std::string and the STL. Mine was different though, after tracking down a memory leak that was happening with the creation of just new empty string, I discovered in the stdlib that there was a shared pointer to the empty string with a reference count of how many…

> you can't trust libraries blindly, even one of the most used and broadly adopted ones There is a corollary to development and debugging. When things break in mysterious ways, we tend to go through a familiar song and dance. As experience, skills and even personal networks grow, we can find ourselves diving ever further in the following chain. 1. "It must be in my code." -- hours of debugging 2. "Okay, it must be so…

Spot on. The experience level of a developer is directly proportional to how fast he/she assumes that the problem is in someone else's code. ;-)

Re: The Horror in the Standard Library

#119

Post by theclaw »

nly wrote: post #76
charles-salvia wrote: post #7

Earlier quoted context omitted.

The C++ delete[] operator doesn't take a size parameter either. This is neither here nor there, and unrelated to the problem the blog post is talking about.

> The C++ delete[] operator doesn't take a size parameter either Yes it does. See overload 6 here http://en.cppreference.com/w/cpp/memory/new/operator_delete

Note that the size parameter is ignored by the standard library implementation. It is intended for use by user-defined implementations.

Re: The Horror in the Standard Library

#120

Post by shabble »

gone35 wrote: post #111
bostik wrote: post #51

Earlier quoted context omitted.

> you can't trust libraries blindly, even one of the most used and broadly adopted ones There is a corollary to development and debugging. When things break in mysterious ways, we tend to go through a familiar song and dance. As experience, skills and even personal networks grow, we can find ourselves diving ever further in the following chain. 1. "It must be in my code." -- hours of debugging 2. "Okay, it must be so…

11. "Do we have the IP core for this?" 12. "Where is my electron microscope?"

..13 "We're gonna need some time on the FIB workstation"[1]

[1] http://www.electronicdesign.com/eda/fib-circuit-edit-becomes...

Post reply on HN