Live data from Hacker News

Dynamic linking

drewdevault.com

121–130 of 249 posts

Re: Dynamic linking

#121

Earlier quoted context omitted.

> I believe that this the reason why static libraries are usually shipped as separate object files (.o) within ar archives (.a), as those were only linked in on demand. Yep. One function per C/obj file for smallest static binary possible.

Can't you achieve the same result with -ffunction-sections?

Yes. And its twin, -fdata-sections.

Re: Dynamic linking

#122

Earlier quoted context omitted.

True, but a couple github imports and 10 lines of code generates a 100mb binary. But, to be fair, I guess we're okay with shipping huge binaries now a days because we're literally shipping whole environments with docker anyway.

To be fair, you're not really supposed to be shipping a full system in a docker image if you can help it; you're supposed to layer your application over the smallest base that will support it (whether that's scratch, distroless, alpine, or a minimal debian base). Of course, I'll be the first to agree that "supposed to" and reality have little in common; if I had a dollar for every time I've seen a final image that st…

In my experience, most who build fat images with the compiler chain and everything do so because they're simply not aware that multi-stage builds are a thing now.

Re: Dynamic linking

#123
post #116
post #66

Earlier quoted context omitted.

This comment hits all of the boxes for everything that is wrong with Hacker News. "First, false statement. Since you know false assumption, I would false conclusion. And of course, this means false conclusion. I'd like to see analysis done on what you did them on." "Second, the arguments seem a little cherry-picked. "Quote from article about W and Z" is cute. But modern systems have a lot of Z, and Obviously You Didn…

I prefer that comment to your reply. By a lot. It attempted to apply reason. You only mocked the form the arguments took. I am firmly of the opinion that content is more important than form. That comment did make a major mistake. Arch Linux is not a source distribution. But the mistake notwithstanding, it was otherwise well-reasoned. Your response was not. You can mock the point that there is still a benefit in shari…

> I am firmly of the opinion that content is more important than form.

This is quite possibly the stupidest thing I have heard this year. How could you possibly have come up with something that brain-dead? Is this how you normally are, or did you have to reach deep into your mind to find the most inane thing you could possibly say and pollute Hacker News with it? Goodness, I hope so. I don't even want to think about the possibility that you are this clueless in real life. I pity those that have to interact with you.

Delivery matters, and content does too. IMO sometimes the information you're conveying can make up for form-many disagree, however, and I'm sure you're aware of this. But there are no absolutes here; sometimes form is more important than content. (And to round out this comment, I thought the one being discussed was fine as I mention in another comment. I'm sometimes accused of being too harsh myself: https://news.ycombinator.com/item?id=23354453)

Re: Dynamic linking

#124
post #110

Earlier quoted context omitted.

The problem with compile times is mostly not the compiler's fault. It is with the project setup. See http://www.real-linux.org.uk/recursivemake.pdf for a classic explanation.

But we haven't learned from this as modern, non-make based build systems still suffer from terribly slow compiles. Take Rust for example. I don't know a single project that uses make to build (they all use cargo), yet Rust suffers from extremely slow compile times. Much of this (as far as I understand) comes from the LLVM compiler, which is why I was picking on compilers.

Rust's slow builds come from rustc, which is based on LLVM. Other LLVM frontends, like clang, are not as slow. This is in part because borrow-checking has a cost.

Re: Dynamic linking

#125

Earlier quoted context omitted.

> First, this analysis was done on Arch Linux, a source-based distribution. Since you know at compile time what your environment is, I would expect the benefits to be smaller. And of course, this means you're willing to do a lot of recompiles. I'd like to see analysis done on more traditional (& common) binary distros. Arch Linux is not Gentoo. And AUR is only a secondary method of installing software. So I'm not sur…

It's not Gentoo, but in practice, many frequently-used packages remain in the AUR and an AUR helper to make that transparent is standard kit for anyone using the box as a workstation. It's much more "source-based" than it may appear at first glance. I have ~15G of self-built packages sitting in my PKGDEST right now, if that counts for anything.

Have you checked if they have official packages yet? I was surprised to find that half my AUR packages already were in official recently. Font packages and Android Studio can easily take up 5G. Also, you don't need an AUR helper.

Re: Dynamic linking

#126
post #51

Static linking allows LTO with aggressive inlining and is therefore able to achieve far superior performance beyond just the startup time. Arguing that dynamic vs. static has better RAM utilization or not is pointless because nowadays we have plenty of RAM but single core performance is stagnating for almost a decade already. Moores law might give us more transistors but single thread performance is still more or les…

lol, you don't want aggressive inlining everywhere.

Here's a good example. Firefox does even cross-language LTO for stuff that matters — all the DOM/CSS/etc components that are tightly bound together. But it happily allows you to dynamically link to a system libjpeg, and you won't lose any performance because the libxullibjpeg boundary is crossed.. about a couple times per downloaded jpeg. Heck, what would you even inline between the jpeg decoder and the browser? Absolutely nothing!

The world at large is already using static linking where it matters.. e.g. with C++, header-only libraries and anything that uses templates results in a lot of static linking. Even if you link to libboost_something.so, you probably already have most of that something inlined into your code because templates :)

Re: Dynamic linking

#127
post #42

Back in the 90s we'd statically link the most frequently executed programs on busy servers for a significant performance boost. Dynamic linking is not a performance feature, it's a decoupling feature.

FreeBSD statically links LLVM into Clang faster build times (2020). (The difference is pretty marginal, but it's at least a percent or two faster.) So my /usr/bin/clang is a 70 megabyte file.

The vast majority of the contents are r-x text (code) and r-- read-only data. I don't know if the kernel is capable of sharing read-only paged memory references or not.

Re: Dynamic linking

#129

Say we have program X with dependency Y. X+Y is either dynamic or static. X can either have responsive maintainers or unresponsive maintainers. Y can either change to fix a bug or change to add a bug. (With Heartbleed, I remember our server was fine because we were on some ancient version of OpenSSL.) Here are the scenarios: - dynamic responsive remove bug: Positive/neutral. Team X would have done it anyway. - dynami…

> I remember our server was fine because we were on some ancient version of OpenSSL.

It's a side point, but I don't entirely understand the argument here since it's basically guaranteed you were vulnerable to a variety of other bugs due to not updating. It depends how old your OpenSSL version was I suppose, but still, it kinda feels like gloating you survived a hurricane by doing no preparation - I'm happy for you, but that doesn't necessarily make it a great idea ;)

> - dynamic responsive remove bug: Positive/neutral. Team X would have done it anyway. > - static responsive remove bug: Positive/neutral: Team X will incorporate the change from Y, although possibly somewhat slower (but safer).

You sure about that? That's basically your whole argument, and I really don't buy it. In my experience, any dependency (static or not) shipped with a program is rarely actively updated, and also never in any regular fashion - on the list of things to do it's typically near the bottom. You called this a neutral, but it's easily a positive for dynamic (and negative for static), and IMO is the biggest argument for dynamic linking and shipping libraries separately. And once you apply that change, it's just two Positives and two Negatives each.

But with that, there's another situation that you're ignoring which throws a big wrench into your table - you're assuming you already know what the dependencies of a program are, when in practice you usually don't. In a dynamically linked world, to address a bug in OpenSSL I just drop a fixed version of OpenSSL in `/lib` and reboot. If it's statically linked to some or all of my programs instead, I need to figure out which if any programs make use of a statically linked version of `OpenSSL` and either wait for a new version or attempt to recompile them with a new one. And if I miss one, then the bug is still there in some form.

Edit: It's a small point, but you're also assuming that the maintainers shipping the static library will catch every bugged version before shipping their program with it - and library maintainers typically try not to release versions with known unusable bugs, it's an accident. An issue like Heartbleed is always going to be found after it's already in the wild, making the `static responsive add bug` category a bit suspect. If a program maintainer was on top of their OpenSSL version and always shipped the latest version when they released, their users would have been vulnerable - they may have made a release in a few days with a new version (to go with the other category), but their users still got the bad OpenSSL version.

Re: Dynamic linking

#130
post #116
post #66

Earlier quoted context omitted.

This comment hits all of the boxes for everything that is wrong with Hacker News. "First, false statement. Since you know false assumption, I would false conclusion. And of course, this means false conclusion. I'd like to see analysis done on what you did them on." "Second, the arguments seem a little cherry-picked. "Quote from article about W and Z" is cute. But modern systems have a lot of Z, and Obviously You Didn…

I prefer that comment to your reply. By a lot. It attempted to apply reason. You only mocked the form the arguments took. I am firmly of the opinion that content is more important than form. That comment did make a major mistake. Arch Linux is not a source distribution. But the mistake notwithstanding, it was otherwise well-reasoned. Your response was not. You can mock the point that there is still a benefit in shari…

the article ignored the few libraries that get shared a lot

The author didn't ignore this, because it's pretty much false. Every package people insisted was relevant, he shot down before he published the article. Example:

https://cmpwn.com/@sir/104406644780241359

Hitchen's razor.

Post reply on HN