Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Horror in the Standard Library

zerotier.com

Post reply on HN
Print view View original

211–220 of 222 posts

Re: The Horror in the Standard Library

#211

Post by jwakely »

mtanski wrote: post #32
kartD wrote: post #20

Earlier quoted context omitted.

Actually from my understanding, it's libstdc++'s allocator that is causing the issue, not malloc.

A big reason the small object optimization exists in libstdc++ containers is because system malloc() is not fast enough. We're not talking about another optimization (small object / locality) as his issue was caused by libstdc++ alloc pools which would not need to exist in the first place if system malloc was better. So libstdc++ reinvents end up reinventing the wheel poorly. As the author mentioned, when he disabled…

By default libstdc++ stopped using the pooling allocator in 2005: https://gcc.gnu.org/r106665

That's one year after the ancient, bitrotted, unofficial copy of the libstdc++ documentation that the blog post links to, but still ancient history.

Re: The Horror in the Standard Library

#212

Post by jwakely »

staticlibs wrote: post #93
arunc wrote: post #54

I encountered exactly the same issue few years ago in UIDAI in one of our large scale biometric matchers and the resolution was exactly the same. After a week of debugging I found that the libstdc++ allocator was the culprit. I found [1] and confirmed the same, which helped in fixing this issue. The thing that was more interesting (or sad) was to know that the GCC developers didn't expect the multithreaded applicatio…

Looks like this is just one of possible "non-default" extension allocators, that can be selected during libstdc++ compile time: https://gcc.gnu.org/onlinedocs/libstdc++/manual/memory.html#...

Facts and up-to-date documentation, how novel.

Re: The Horror in the Standard Library

#213

Post by jwakely »

api wrote: post #23
cyphar wrote: post #2

Did you report the issue upstream with a patch? The solution to "the standard library is broken" is to fix the standard library, no? It's all free software after all.

It's a known issue according to what I've read. It's never been fixed. This may be due to the fact that it's hard to trigger and reproduce.

It's not a known issue, but GLIBCPP_FORCE_NEW has had no effect on libstdc++ code for more than a decade, so I wonder which prehistoric version you're using. Even the modern GLIBCXX_FORCE_NEW doesn't do anything for the default std::allocator implementation, which always uses new/delete unconditionally (since 2005).

Re: The Horror in the Standard Library

#214

Post by jwakely »

IshKebab wrote: post #73
cyphar wrote: post #55

Earlier quoted context omitted.

They make the case that it's broken with very strong language. They don't say "here is a patch" or even an outline of a patch (or a link to a bug report even), just histrionics. > GNU libstdc++ is broken. This is pretty unforgiveable. [...] Adding wheels to the wheel is sometimes forgiveable when dealing with closed systems that you can't fix but libstdc++ and glibc are both open source GNU projects. I think there's…

You can certainly send a patch, but there's no guarantee they'll accept it. Some core projects seem to be highly opinionated. Look at GCC's attitude to plugins or good error messages. Or Linux and gr-security. Or Linux and stable driver ABIs. There are plenty of things that people would like to change but can't because the maintainers disagree. Not saying that is necessarily bad but you are stupid if you think the an…

> Look at GCC's attitude to plugins or good error messages.

Which is?

A patch isn't necessary for it to be fixed, but a bug report generally is. A blog post linking to unofficial copies of documentation from 2004 doesn't count.

Re: The Horror in the Standard Library

#216

Post by fnord123 »

MichaelBurge wrote: post #31
halayli wrote: post #21

this conclusion might be wrong. the code in question while it might not be allocating/freeing memory it might be stumbling on memory blocks and corrupting mem management structures. Turning the flag on might be fixing the issue by mere luck because memory allocations, locations and structures would be different

Valgrind would've caught those, wouldn't it? Or, maybe the C++ layer prevents it from catching that since it's application-level, which is at least another reason to remove it.

The tools are good but not magic. It's always possible that they miss something (e.g. when multithreading, the bug might not manifest since the timings are all different when running in valgrind). But this is a red flag:

"Nothing worked. It's leaking but it's not. It's leaking but the debugger says no memory was lost. It's leaking in ways that are dependent on irrelevant changes to the ordering of mundane operations. This can't be happening."

This is a red flag for heap corruption - or multithreading bugs. (Stack corruption is usually a crash and a wrong stack trace). As it's not trivially reproducible, it's probably a multithreading bug. It's also easy to imagine that OP wrote a scripty-input generator in C++ to run through valgrind which gives single threaded inputs. So running under valgrind it won't be detected. So it's never fun to solve and ever since I changed languages, my multithreaded debugging skills have become a bit rusty. Hey-o!

But it would be good if he amended his post to reduce the vitriol aimed at GCC. It's demonstrably false that GCC's default allocator holds a cache. It makes OP look stupid to people in the know; and makes GCC look bad to people not in the know. No one wins.

Re: The Horror in the Standard Library

#217

Post by Beltiras »

bostik wrote: post #51
bluejekyll wrote: post #29

OMG, as I was reading this I thought, "man, this reminds me of a bug I ran into with std::string back in 2000", A few sentences later, and this is also about std::string and the STL. Mine was different though, after tracking down a memory leak that was happening with the creation of just new empty string, I discovered in the stdlib that there was a shared pointer to the empty string with a reference count of how many…

> you can't trust libraries blindly, even one of the most used and broadly adopted ones There is a corollary to development and debugging. When things break in mysterious ways, we tend to go through a familiar song and dance. As experience, skills and even personal networks grow, we can find ourselves diving ever further in the following chain. 1. "It must be in my code." -- hours of debugging 2. "Okay, it must be so…

This is the most concise list of debugging layers I have seen. I'm mostly commenting so that I get this into my comment feed and can locate it easier.

Re: The Horror in the Standard Library

#218

Post by bostik »

Beltiras wrote: post #217
bostik wrote: post #51

Earlier quoted context omitted.

> you can't trust libraries blindly, even one of the most used and broadly adopted ones There is a corollary to development and debugging. When things break in mysterious ways, we tend to go through a familiar song and dance. As experience, skills and even personal networks grow, we can find ourselves diving ever further in the following chain. 1. "It must be in my code." -- hours of debugging 2. "Okay, it must be so…

This is the most concise list of debugging layers I have seen. I'm mostly commenting so that I get this into my comment feed and can locate it easier.

In that case, feel free to bookmark this one: http://bostik.iki.fi/aivoituksia/random/developer-debugging-...

This got more attention that I thought possible, so I decided to pull it out as an item all by itself.

Re: The Horror in the Standard Library

#219

Post by kosma »

northern_lights wrote: post #135
kosma wrote: post #89

Earlier quoted context omitted.

> 10. "Where is my chip lab grade oscilloscope?" 11. "Shit, where do I borrow a spectrum analyzer and a set of near field probes? These things cost an arm and a leg!" Yes, STM32F1 MCUs generate inference that jams GPS receivers. No, it's not documented anywhere.

any MCU can "jam" a GPS receiver if the board is laid out improperly or without enough shielding

A bare, free-floating STM32F103 with literally nothing but a LiPo battery connected with two wires, running the blinky.c demo, will completely jam many GPS receivers when placed next to the antenna.

Re: The Horror in the Standard Library

#220

Post by kosma »

makomk wrote: post #151
kosma wrote: post #89

Earlier quoted context omitted.

> 10. "Where is my chip lab grade oscilloscope?" 11. "Shit, where do I borrow a spectrum analyzer and a set of near field probes? These things cost an arm and a leg!" Yes, STM32F1 MCUs generate inference that jams GPS receivers. No, it's not documented anywhere.

You'd be surprised how much you can find out just with a $10 TV tuner dongle and a piece of coax with a short section of the outer braid trimmed off at once end.

RTL-SDR is the Arduino of EMC work. :)
Post reply on HN