Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dynamic linking

drewdevault.com

Post reply on HN
Print view View original

71–80 of 249 posts

Re: Dynamic linking

#71

Post by jart »

pengaru wrote: post #69

Static linking has been such a nuisance for the libSDL folks that they implemented dynamic loading of itself [0], controlled via an environment variable, as an escape hatch from executables w/libSDL linked statically. It's understandable that games, especially proprietary ones, distribute statically-linked binaries ensuring any third-party dependencies will be present and be a compatible version. But the value of tha…

There's nothing wrong with taking a principled approach to builds. For example, it's not just a question of static or dynamic. Linking of vendored/static/pinned/etc. sources/artifacts/etc. in general comes with the benefit of hermeticity. That enables tools to work better, since it gives them perfect knowledge of the things that exist. It also entails a certain degree of responsibility for writing your own scripts that can generate automated alerts should issues arise in the upstream sources.

Re: Dynamic linking

#72

Post by kccqzy »

AshamedCaptain wrote: post #9

First we claim that dynamic linking does not provide any memory savings because the libc we used is small, and later on we use our lack of dynamic linking to justify having a small libc. Smart, very smart.

That's what happens when you decouple components and make decisions for each component separately rather than holistically for the system as a whole.

Re: Dynamic linking

#73

Post by bjourne »

setr wrote: post #24

I believe the original reasoning for Dynamic Linking wasn't performance gains, but security gains -- someone described the driving story to me as essentially a found vulnerability in a very common library required updating and re-compiling everything on every system , scarring sysadmins globally and permanently; the space saving and performance aspects came up as later "bonuses". I have little memory of the details o…

Stallman and other GNU folks have claimed that they stuck with dynamic linking to foster cooperation between hackers. They knew that it wasn't a win performance-wise but they wanted to encourage hackers to help each other. The idea was that if someone found a bug in a library he or she depended on they would be "forced" to send the patch upstream rather than just fixing the bug in their local copy of the library. Thus they made dynamic linking the default in gcc and kept static linking as something of an after though.

I read it on a mailing list a long time ago so I don't have a source.

Re: Dynamic linking

#74

Post by kccqzy »

k__ wrote: post #47
cryptonector wrote: post #35

Earlier quoted context omitted.

https://news.ycombinator.com/item?id=23656173

Are these also issues for languages like Go and Rust?

Of course not. It's not even a problem for C++ if you use namespaces pervasively. This is purely support for C.

Re: Dynamic linking

#75

Post by old-gregg »

ddevault wrote: post #13
old-gregg wrote: post #10

Earlier quoted context omitted.

Run htop or similar, sort by "shared memory" column and see how much more memory you'd need per process if shared linking did not exist. I think the author's using a wrong method to make a point. Dynamic linking feels out of place for most long-running server-side apps (typical SaaS workload). One can argue that in a mostly CLI-environment there's also not much benefit. But even an empty Ubuntu desktop runs ~400 proc…

That is an extremely misleading figure. Shared memory is page-aligned entire libraries dropped into RAM. Statically linking would, as the article shows, only use on average about 4% of the symbols available from the libraries, and the majority of this would not end up in RAM with your statically linked binary. And if you used a more selective approach, dynamically linking to no more than perhaps a dozen high-impact l…

That is an extremely misleading figure. Shared memory is page-aligned entire libraries dropped into RAM

First of all, I seriously doubt it (haven't looked closely, but if library-loading is similar to mmap, it should only count actually used segments).

But that shouldn't even matter, as most of these libraries are fully utilized. All of libc is used collectively by 400+ processes, that is also true for the complex multi-layer GUI machinery. The output of ldd $(which gnome-calculator) is terrifying, run it under a profiler and see how many functions get hit, you'll be amazed.

Put the cold hard numbers right in front of someone's face and still the cargo cult wins out.

The coldness of your numbers did not impress. And calling a reasonable engineering trade-off "cargo cult" doesn't get you any points either.

Static linking is better than dynamic linking. Sometimes. And vice versa. That's how engineering is different from science, there are no absolute truths, only trade-offs.

Re: Dynamic linking

#76

Post by a1369209993 »

devit wrote: post #16
weinzierl wrote: post #12

Earlier quoted context omitted.

> A better way to do this analysis would be to build a Linux distribution with everything statically linked [..] Here you go: Stali https://dl.suckless.org/htmlout/sta.li/ "Stali distribution smashes assumptions about Linux" https://www.infoworld.com/article/3048737/stali-distribution...

For a useful comparison you need the static and dynamic distributions to be otherwise the same, i.e. you want to pick a mainstream distribution and build it from scratch with both static and dynamic linking and compare.

I don't know of any mainstream distribution that doesn't make full-static-from-scratch builds gratuitously painful and difficult. Personally I gave up after trying to blunt-force-trauma glibc into linking correctly, though, so someone with more internals knowledge might have better success at it.

Re: Dynamic linking

#77

Post by saagarjha »

kick wrote: post #66
dwheeler wrote: post #28

Not convinced. First, this analysis was done on Arch Linux, a source-based distribution. Since you know at compile time what your environment is, I would expect the benefits to be smaller. And of course, this means you're willing to do a lot of recompiles. I'd like to see analysis done on more traditional (& common) binary distros. Second, the arguments seem a little cherry-picked. "Over half of your libraries are us…

This comment hits all of the boxes for everything that is wrong with Hacker News. "First, false statement. Since you know false assumption, I would false conclusion. And of course, this means false conclusion. I'd like to see analysis done on what you did them on." "Second, the arguments seem a little cherry-picked. "Quote from article about W and Z" is cute. But modern systems have a lot of Z, and Obviously You Didn…

I know it's a popular Hacker News trope, but when done well I actually appreciate comments of the style "the author does a comparison and has some results, but do also keep in mind xyz that is non-obvious". I know there is a problem with people who go through articles and try to pick things that could affect the results and treat that as invalidating them, but more often than not simple comparisons like these do have more to say about them than what is presented and I don't want to discourage those.

Re: Dynamic linking

#78

Post by a1369209993 »

old-gregg wrote: post #10
k__ wrote: post #6

What are the counter arguments?

Run htop or similar, sort by "shared memory" column and see how much more memory you'd need per process if shared linking did not exist. I think the author's using a wrong method to make a point. Dynamic linking feels out of place for most long-running server-side apps (typical SaaS workload). One can argue that in a mostly CLI-environment there's also not much benefit. But even an empty Ubuntu desktop runs ~400 proc…

> Run htop or similar, sort by "shared memory"

The top two entries are 80 and 36 kB respectively. RES is 2.3 giga-bytes (over four orders of magnitude larger) between them. Even multiplying the top SHR by your ~400 processes gives 32 MB (still two orders of magnitude off). That is not a counter argument; that is a agreement that dynamic linking is useless.

Edit: RES, not VIRT.

Re: Dynamic linking

#79

Post by jeffbee »

binarycrusader wrote: post #65
pengaru wrote: post #42

Back in the 90s we'd statically link the most frequently executed programs on busy servers for a significant performance boost. Dynamic linking is not a performance feature, it's a decoupling feature.

It is a performance feature if you are memory constrained. Shared libraries are “shared” for a reason. On a server with high multi-tenancy the savings can be significant.

Not really. Dynlinking solves a memory resource problem we had in the 80s. These days the only people with the problem are the very smallest embedded systems.

As for "high multi-tenancy" there's nobody out there with server occupancy as high as Google's (see the recent Borg scheduler traces for concrete data) and they statically link everything.

Re: Dynamic linking

#80

Post by arcticbull »

iforgotpassword wrote: post #53
Vogtinator wrote: post #4

> On average, dynamically linked executables use only 4.6% of the symbols on offer from their dependencies. That's correct, but also very misleading and leads to the wrong conclusion. The dynamically linked library has references to itself, externally visible or not. It would be wrong to claim that Application.run(); only uses a single symbol of a library. > A good linker will remove unused symbols. With LTO or -f{fu…

> I believe that this the reason why static libraries are usually shipped as separate object files (.o) within ar archives (.a), as those were only linked in on demand. Yep. One function per C/obj file for smallest static binary possible.

Can't you achieve the same result with -ffunction-sections?
Post reply on HN