Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dynamic linking

drewdevault.com

Post reply on HN
Print view View original

111–120 of 249 posts

Re: Dynamic linking

#111

Post by jart »

andoma wrote: post #26
jnwatson wrote: post #17

Earlier quoted context omitted.

Go executables are statically linked. It makes deployment a breeze. I think you overestimate how much saving you get from dynamically linking libc. Each executable uses only a small portion of libc, so the average savings is going to be in the handful of kilobytes per executable.

In theory yes. However, in practice static linking with glibc pulls in a lot of dead weight, musl comes to the rescue though: test.c: int main(int argc, char **argv) { printf("hello world\n"); return 0; } Dynamic linking (glibc): $ gcc -O2 -Wl,--strip-all test.c $ ls -sh a.out 8.0K a.out Static linking (glibc): $ gcc -O2 --static -Wl,--strip-all test.c $ ls -l a.out 760K a.out Static linking (musl): $ musl-gcc --stat…

Static linking (https://github.com/jart/cosmopolitan)

    jart@debian:~/cosmo$ make -j12 CPPFLAGS+=-DIM_FEELING_NAUGHTY MODE=tiny o/tiny/examples/hello.com
    jart@debian:~/cosmo$ ls -sh o/tiny/examples/hello.com
    20K o/tiny/examples/hello.com
Note: Output binary runs on Windows, Mac, and BSD too.

Re: Dynamic linking

#112

Post by zelly »

Dynamic linking provides encapsulation and security benefits. An application with a statically linked OpenSSL can be vulnerable if a CVE comes out for that version of OpenSSL, whereas a dynamically linked OpenSSL could be patched immediately without recompilation (which may be impossible if the software is proprietary). The vendor of the shared library can update the implementation without requiring all downstream consumers to recompile. This should not be downplayed.

Pure static linking makes sense when you are deploying code you control onto an environment you control.

While I like ease of deploying Go, where binaries are one big blob that just works, it makes it closer to the Java style than the UNIX modular-tool-does-one-thing-well tradition.

Re: Dynamic linking

#113

Post by [deleted] »

kick wrote: post #66
dwheeler wrote: post #28

Not convinced. First, this analysis was done on Arch Linux, a source-based distribution. Since you know at compile time what your environment is, I would expect the benefits to be smaller. And of course, this means you're willing to do a lot of recompiles. I'd like to see analysis done on more traditional (& common) binary distros. Second, the arguments seem a little cherry-picked. "Over half of your libraries are us…

This comment hits all of the boxes for everything that is wrong with Hacker News. "First, false statement. Since you know false assumption, I would false conclusion. And of course, this means false conclusion. I'd like to see analysis done on what you did them on." "Second, the arguments seem a little cherry-picked. "Quote from article about W and Z" is cute. But modern systems have a lot of Z, and Obviously You Didn…

[deleted]

Re: Dynamic linking

#114

Post by eikenberry »

btilly wrote: post #110
eikenberry wrote: post #63

Earlier quoted context omitted.

If compilers weren't so pathetically slow this wouldn't be that much of an issue. If this became widespread it might have the positive impact on projects like LLVM and get them to pay some attention to compiling time, not just optimization.

The problem with compile times is mostly not the compiler's fault. It is with the project setup. See http://www.real-linux.org.uk/recursivemake.pdf for a classic explanation.

But we haven't learned from this as modern, non-make based build systems still suffer from terribly slow compiles. Take Rust for example. I don't know a single project that uses make to build (they all use cargo), yet Rust suffers from extremely slow compile times. Much of this (as far as I understand) comes from the LLVM compiler, which is why I was picking on compilers.

Re: Dynamic linking

#115

Post by cookiecaper »

abjKT26nO8 wrote: post #39
dwheeler wrote: post #28

Not convinced. First, this analysis was done on Arch Linux, a source-based distribution. Since you know at compile time what your environment is, I would expect the benefits to be smaller. And of course, this means you're willing to do a lot of recompiles. I'd like to see analysis done on more traditional (& common) binary distros. Second, the arguments seem a little cherry-picked. "Over half of your libraries are us…

> First, this analysis was done on Arch Linux, a source-based distribution. Since you know at compile time what your environment is, I would expect the benefits to be smaller. And of course, this means you're willing to do a lot of recompiles. I'd like to see analysis done on more traditional (& common) binary distros. Arch Linux is not Gentoo. And AUR is only a secondary method of installing software. So I'm not sur…

It's not Gentoo, but in practice, many frequently-used packages remain in the AUR and an AUR helper to make that transparent is standard kit for anyone using the box as a workstation. It's much more "source-based" than it may appear at first glance. I have ~15G of self-built packages sitting in my PKGDEST right now, if that counts for anything.

Re: Dynamic linking

#116

Post by btilly »

kick wrote: post #66
dwheeler wrote: post #28

Not convinced. First, this analysis was done on Arch Linux, a source-based distribution. Since you know at compile time what your environment is, I would expect the benefits to be smaller. And of course, this means you're willing to do a lot of recompiles. I'd like to see analysis done on more traditional (& common) binary distros. Second, the arguments seem a little cherry-picked. "Over half of your libraries are us…

This comment hits all of the boxes for everything that is wrong with Hacker News. "First, false statement. Since you know false assumption, I would false conclusion. And of course, this means false conclusion. I'd like to see analysis done on what you did them on." "Second, the arguments seem a little cherry-picked. "Quote from article about W and Z" is cute. But modern systems have a lot of Z, and Obviously You Didn…

I prefer that comment to your reply. By a lot. It attempted to apply reason. You only mocked the form the arguments took. I am firmly of the opinion that content is more important than form.

That comment did make a major mistake. Arch Linux is not a source distribution. But the mistake notwithstanding, it was otherwise well-reasoned. Your response was not.

You can mock the point that there is still a benefit in sharing a library a few times, and the article ignored the few libraries that get shared a lot. But said point remains true.

You may think that Android isn't relevant to the article's point. But the tradeoffs between static and dynamic linking are true across operating systems. The challenges that Android has had because of static linking are therefore worth paying attention to. Refusing to look for parallels to inform your intuition from is simply refusal to learn.

Re: Dynamic linking

#117

Post by btilly »

eikenberry wrote: post #114
btilly wrote: post #110

Earlier quoted context omitted.

The problem with compile times is mostly not the compiler's fault. It is with the project setup. See http://www.real-linux.org.uk/recursivemake.pdf for a classic explanation.

But we haven't learned from this as modern, non-make based build systems still suffer from terribly slow compiles. Take Rust for example. I don't know a single project that uses make to build (they all use cargo), yet Rust suffers from extremely slow compile times. Much of this (as far as I understand) comes from the LLVM compiler, which is why I was picking on compilers.

I have not used Rust very much.

But in https://doc.rust-lang.org/cargo/reference/build-scripts.html I find comments like, It is recommended to carefully consider each dependency you add, weighing against the impact on compile time, licensing, maintenance, etc. Cargo will attempt to reuse a dependency if it is shared between build dependencies and normal dependencies. However, this is not always possible, for example when cross-compiling, so keep that in consideration of the impact on compile time.

So it looks like cargo can have the same issue that make does with recursive dependencies.

Re: Dynamic linking

#118

Post by loeg »

mforney wrote: post #41
loeg wrote: post #27

Earlier quoted context omitted.

Do you static-link Linux? :-)

I'm not sure what you mean here. Are you asking whether I build my kernel drivers as modules or built-in? Personally, I build my kernels without modules, but I've never heard of that technique being called "static-linking Linux".

Yes, exactly. Loadable modules are a form of dynamic linking. I've never heard it called "static-linking Linux," either, but I think static / dynamic linking is well understood, and I'm happy you understood the meaning from context.

Re: Dynamic linking

#119

Post by loeg »

bjourne wrote: post #62
loeg wrote: post #25

Earlier quoted context omitted.

> Put the cold hard numbers right in front of someone's face and still the cargo cult wins out. Come on. You did not actually measure the figure GP mentioned and which you are disputing. Your methodology and assumption — that 4% external symbol use translates into 4% size used — is a plausible guess, but you haven't supported it with data. Even if you had measured the figure you're accusing GP of ignoring, the tone o…

> There is a good reason to page- or superpage-align code generally; it burns some virtual memory but reduces TLB overhead and therefore misses / invalidations, which are very costly. You would want to do the same with executable code in a static-linked binary. But most code isn't performance critical. Thus trying to align functions to page boundaries is just wasting memory. Even in performance critical code, alignin…

> But most code isn't performance critical.

Wasting TLB slots on your unimportant code still pessimizes your hot code.

> Thus trying to align functions to page boundaries is just wasting memory.

No one aligns individual functions to page boundaries; you align the entire loadable code segment to a page (or preferably, superpage) boundary.

> Even in performance critical code, aligning to cache line sizes is enough and aligning to page boundaries doesn't provide any advantage.

This is a different kind of optimization (avoiding cache line contention) than I was talking about (optimizing TLB slot use).

Yes, forced page-alignment doesn't help cache line contention anymore than forced cacheline-alignment. But that's irrelevant for code, generally: (outside of self-modifying code, which is extremely uncommon) code doesn't share a cacheline with memory that will be mutated and thus doesn't contend in that way.

Re: Dynamic linking

#120

Post by loeg »

MaulingMonkey wrote: post #105
bjourne wrote: post #62

Earlier quoted context omitted.

> There is a good reason to page- or superpage-align code generally; it burns some virtual memory but reduces TLB overhead and therefore misses / invalidations, which are very costly. You would want to do the same with executable code in a static-linked binary. But most code isn't performance critical. Thus trying to align functions to page boundaries is just wasting memory. Even in performance critical code, alignin…

Not only is it wasting memory, but I wouldn't be convinced it'd not be outright hurting performance by increasing cache line aliasing, increasing TLB overhead and misses/invalidations! Avoiding page alignment can be a performance gain! https://pvk.ca/Blog/2012/07/30/binary-search-is-a-pathologic... Like, sure, mapping your executable's code section in at a page boundary is probably fine, but I think trying to align i…

> I think trying to align individual functions to page boundaries would be a counterproductive mistake as a general strategy.

Yes — which is why no one does that. I don't know where bjourne came up with the idea.

Post reply on HN