Delete an inline function, save 794 kB • Hacker News Board

Re: Delete an inline function, save 794 kB

#41

Post by brucedawson » Mon, Jan 23, 2017, 11:28 PM UTC

vetinari wrote: post #16

valarauca1 wrote: post #11

Earlier quoted context omitted.

I have in fact written my own ELF parser. I was speaking more to static libraries/binaries which once parsed are only a handful of pointer jumps/memset calls. Dynamic segments are very complex :| The load time for Dynamic libraries isn't really on the _loader_. You need to do disk searches/file IO to load the libraries parsing MANY additional files. Starting a program that has 20 dynamic links isn't starting a progra…

There is another thing that slows down the symbol resolution: the symbols are not scoped per library or soname, they are global and so every symbol is searched in every linked library. Link few C++ libraries, the symbol count explodes and the runtime linking slows downs. Large C++ apps, like OpenOffice.org, used to have a problem with this. The I/O may or may not be a bottleneck. The libraries are being mmaped and if…

Whenever I hear about how loaders resolve symbols on Linux I am amazed that it works at all, and that it runs reasonably fast.

The loader on Windows has a much simpler job - it imports specific symbols from specific DLLs. In many cases the importing is done by ordinal instead of by name, to make it even faster.

I'm used to the Windows system so I don't miss the extra functionality that the Linux loader provides - in fact I find it scary.

Re: Delete an inline function, save 794 kB

#42

Post by the_why_of_y » Mon, Jan 23, 2017, 11:47 PM UTC

JoeAltmaier wrote: post #30

geofft wrote: post #29

Earlier quoted context omitted.

The problem is that the glibc loader has to look through each of those 170 libraries for every single symbol . On Windows, OS X, Solaris, and probably lots of other OSes, the executable format supports saying "This function needs g_string_append from libglib-2.0.so.0" and "That function needs SDL_Init from libsdl-1.2.so.0". The GNU dynamic linker (and the BSD dynamic linker, incidentally) only supports saying "Hi, I'…

Isn't it a feature that you can replace an implementation at link time? E.g. implement your own g_string_append or whatever. If all symbols were bound to libraries, that would be impossible.

It is a feature that is useful for many standard C library functions, a few functions in other commonly used libraries, and next to no functions in application libraries in cases like the parent's OOo example.

A reasonable approach to designing shared libraries would be to allow interposition as a non-default feature for explicitly marked symbols, and use fast direct binding by default.

Re: Delete an inline function, save 794 kB

#43

Post by geofft » Mon, Jan 23, 2017, 11:48 PM UTC

acqq wrote: post #38

geofft wrote: post #29

Earlier quoted context omitted.

The problem is that the glibc loader has to look through each of those 170 libraries for every single symbol . On Windows, OS X, Solaris, and probably lots of other OSes, the executable format supports saying "This function needs g_string_append from libglib-2.0.so.0" and "That function needs SDL_Init from libsdl-1.2.so.0". The GNU dynamic linker (and the BSD dynamic linker, incidentally) only supports saying "Hi, I'…

> It was rejected out-of-hand by Ulrich Drepper That explains quite a bit. Even without him I still consider a lot of Linux development attitude to be "har, har, the noob doesn't know that he has to hexagonize the fongebangler in the WERTR_FREW file before he can use the Backspace key, no we aren't going to change the defaults to match the 99.9999% of the keyboards of the world." I never understood how they enjoy hex…

It's a poisonous culture, where it's more important to look smart than to be smart. Note that Ulrich's answer was not just completely self-assured but also technically incorrect, that he didn't respond to the polite questions about his answer, and that prelink eventually turned out to be a big misfeature.

I suspect they're quietly pentagonizing all the fongebanglers every time they install their systems, confused why there are only five points instead of six, and too worried about their reputation to suggest that maybe someone should make fongebanglers automatically figure out the right polygon, because then they'd they'll admit they don't know actually how to fix it themselves.

Re: Delete an inline function, save 794 kB

#44

Post by acqq » Tue, Jan 24, 2017, 12:15 AM UTC

geofft wrote: post #43

acqq wrote: post #38

Earlier quoted context omitted.

> It was rejected out-of-hand by Ulrich Drepper That explains quite a bit. Even without him I still consider a lot of Linux development attitude to be "har, har, the noob doesn't know that he has to hexagonize the fongebangler in the WERTR_FREW file before he can use the Backspace key, no we aren't going to change the defaults to match the 99.9999% of the keyboards of the world." I never understood how they enjoy hex…

It's a poisonous culture, where it's more important to look smart than to be smart. Note that Ulrich's answer was not just completely self-assured but also technically incorrect, that he didn't respond to the polite questions about his answer, and that prelink eventually turned out to be a big misfeature. I suspect they're quietly pentagonizing all the fongebanglers every time they install their systems, confused why…

Yes, thanks, only his answer to the post you linked to is more than enough to demonstrate how wrong it was allowing him to be responsible for anything. How many years did he manage to block the improvements?

To get back to the topic we discuss: the patch sent there (and rejected by UD) speeds up loading everywhere glibc is used! And that's the pure CPU speedup, no matter how fast SSD you have!

Reading the question of Michael Meeks, it was just a proof of concept, maybe the whole mechanism can be improved even more, he comments his own work:

"Is there a better way to achieve what I do in dl-deps.c? the umpteen-string compares are clearly highly evil"

Anybody knows the status of uclibc? How do they do it? If nobody knows, anybody willing to measure how much the same version of OO needs on the same hardware with glibc and uclibc?

Re: Delete an inline function, save 794 kB

#45

Post by oso2k » Thu, Jan 26, 2017, 7:17 AM UTC

geofft wrote: post #31

oso2k wrote: post #25

Earlier quoted context omitted.

Google's made their attempt a few years ago. It's called GoLD, or, gold [0]. However, its key benefit is in lower link time processing of object files and libraries. Important if you're building apps that take minutes or hours to build every day in a CI/CD system. Another example is browser source branches (10s of 1000s object files, libraries, and objects). Key innovation was to not use GNU's BFD [1]. BFD is a littl…

gold is a good linker for the existing formats, but there are a number of systematic improvements that can be made, ranging from -Bdirect / two-level namespace support (see my other comment) to just rethinking how programs get compiled and linked in the first place. Rust, for instance, bypasses a lot of this complexity by not supporting things like .o files: an entire set of Rust source files is compiled at once, so…

I really wish we had TP-style modules [0] information in our object file formats. What seems so simple conceptually is nearly impossible to replicate?

Can't figure out why C, C++, Java or C# never got there and why JS is puking all over the concept with half-baked, crappy syntactic look-alike (but not work work-alike) modules. Ughh...

[0] http://prog21.dadgum.com/47.html