Dynamic linking • Hacker News Board

Re: Dynamic linking

#141

Post by tomjakubowski » Sat, Jun 27, 2020, 12:27 AM UTC

kick wrote: post #130

btilly wrote: post #116

Earlier quoted context omitted.

I prefer that comment to your reply. By a lot. It attempted to apply reason. You only mocked the form the arguments took. I am firmly of the opinion that content is more important than form. That comment did make a major mistake. Arch Linux is not a source distribution. But the mistake notwithstanding, it was otherwise well-reasoned. Your response was not. You can mock the point that there is still a benefit in shari…

the article ignored the few libraries that get shared a lot The author didn't ignore this, because it's pretty much false. Every package people insisted was relevant, he shot down before he published the article. Example: https://cmpwn.com/@sir/104406644780241359 Hitchen's razor.

> Of the 2188 packages I have installed, 70 of them depend on OpenSSL.

This must not be counting packages which depend on openssl by a chain of dependencies, some number of which contain binaries or libraries which link libssl. On my desktop Arch system of 1448 packages, 42 depend directly on openssl and 672 depend on it directly or indirectly. (pactree -ur openssl | wc -l, then subtract one for openssl itself)

Re: Dynamic linking

#142

Post by vlovich123 » Sat, Jun 27, 2020, 12:27 AM UTC

zelly wrote: post #112

Dynamic linking provides encapsulation and security benefits. An application with a statically linked OpenSSL can be vulnerable if a CVE comes out for that version of OpenSSL, whereas a dynamically linked OpenSSL could be patched immediately without recompilation (which may be impossible if the software is proprietary). The vendor of the shared library can update the implementation without requiring all downstream co…

From distributions where you build everything from source (most cloud providers today, mobile phone operators, etc), such advantages are purely theoretical & never play out in practice. In such environments you're pushing out updates of the entire system on whatever your regular cadence is. Additionally, because the old library is still mapped & running you have to know to restart all the processes that have the old version linked in. Not an easy task.

About the only place it kind of matters is Linux distros where you are deploying closed source binaries linked against system libraries. I'm not sure that's a significant use-case.

The only performance benefit is memory. If you have a core infrastructure library that's widely shared (e.g. libopenssl) then it can have some benefit. In practice I much prefer the microservice model with a formal IPC API. Then the SW update is trivial to fix the exploit - just kill the 1 process providing the service.

Re: Dynamic linking

#143

Post by steveklabnik » Sat, Jun 27, 2020, 12:37 AM UTC

loeg wrote: post #124

eikenberry wrote: post #114

Earlier quoted context omitted.

But we haven't learned from this as modern, non-make based build systems still suffer from terribly slow compiles. Take Rust for example. I don't know a single project that uses make to build (they all use cargo), yet Rust suffers from extremely slow compile times. Much of this (as far as I understand) comes from the LLVM compiler, which is why I was picking on compilers.

Rust's slow builds come from rustc, which is based on LLVM. Other LLVM frontends, like clang, are not as slow. This is in part because borrow-checking has a cost.

(But only a very, very small part, generally.)

Re: Dynamic linking

#144

Post by steveklabnik » Sat, Jun 27, 2020, 12:40 AM UTC

btilly wrote: post #117

eikenberry wrote: post #114

Earlier quoted context omitted.

But we haven't learned from this as modern, non-make based build systems still suffer from terribly slow compiles. Take Rust for example. I don't know a single project that uses make to build (they all use cargo), yet Rust suffers from extremely slow compile times. Much of this (as far as I understand) comes from the LLVM compiler, which is why I was picking on compilers.

I have not used Rust very much. But in https://doc.rust-lang.org/cargo/reference/build-scripts.html I find comments like, It is recommended to carefully consider each dependency you add, weighing against the impact on compile time, licensing, maintenance, etc. Cargo will attempt to reuse a dependency if it is shared between build dependencies and normal dependencies. However, this is not always possible, for example…

This is talking about very different issues than the ones identified in that paper. Most of that paper, if not all, does not directly apply to Cargo.

Re: Dynamic linking

#145

Post by btilly » Sat, Jun 27, 2020, 12:57 AM UTC

saagarjha wrote: post #123

btilly wrote: post #116

Earlier quoted context omitted.

I prefer that comment to your reply. By a lot. It attempted to apply reason. You only mocked the form the arguments took. I am firmly of the opinion that content is more important than form. That comment did make a major mistake. Arch Linux is not a source distribution. But the mistake notwithstanding, it was otherwise well-reasoned. Your response was not. You can mock the point that there is still a benefit in shari…

> I am firmly of the opinion that content is more important than form. This is quite possibly the stupidest thing I have heard this year. How could you possibly have come up with something that brain-dead? Is this how you normally are, or did you have to reach deep into your mind to find the most inane thing you could possibly say and pollute Hacker News with it? Goodness, I hope so. I don't even want to think about…

This is quite possibly the stupidest thing I have heard this year. How could you possibly have come up with something that brain-dead?

That is an opinion. It is your opinion. I happen to have a very different opinion. It may be surprising to you, but surprise doesn't mean that it is trivially wrong.

Experience has taught me that if you have good content, then you have something worthwhile to say if I can get past the form. But, by contrast, someone who only has good form has nothing worthwhile to say. The more I you dig in, the less value that there is to find.

Obviously having both form and content is better than only having one. But if I have to choose one, I will choose to surround myself with people who have good content. Every time.

Is this how you normally are, or did you have to reach deep into your mind to find the most inane thing you could possibly say and pollute Hacker News with it? Goodness, I hope so. I don't even want to think about the possibility that you are this clueless in real life. I pity those that have to interact with you.

If you're curious, you can always find out more about me. My online profile goes back quite far. A few random things that I've written include https://www.perlmonks.org/?node_id=59345, http://elem.com/~btilly/effective-ab-testing/, and the answers at https://stackoverflow.com/users/585411/btilly?tab=answers.

Or you can go with your already formed opinion. It is no skin off my back.

Re: Dynamic linking

#146

Post by zelly » Sat, Jun 27, 2020, 1:48 AM UTC

vlovich123 wrote: post #142

zelly wrote: post #112

Dynamic linking provides encapsulation and security benefits. An application with a statically linked OpenSSL can be vulnerable if a CVE comes out for that version of OpenSSL, whereas a dynamically linked OpenSSL could be patched immediately without recompilation (which may be impossible if the software is proprietary). The vendor of the shared library can update the implementation without requiring all downstream co…

From distributions where you build everything from source (most cloud providers today, mobile phone operators, etc), such advantages are purely theoretical & never play out in practice. In such environments you're pushing out updates of the entire system on whatever your regular cadence is. Additionally, because the old library is still mapped & running you have to know to restart all the processes that have the old…

Suppose 100 processes on the same box call the same function in glibc. If each process had its own redundant version of it mapped into memory, then that's 100x more cache misses compared to them all sharing the same page.

I get that only a handful of libraries are actually used, but that's a very fat tail and the performance will be very bad if that handful of libraries are always statically linked. This may be less important if the program is in a container (isn't sharing anyway) or launched as an AWS Lambda throwaway. I dunno, maybe dynamic linking isn't relevant anymore now that most Linux programs are meant to run in containers on an expensive cloud with tons of memory.

> In practice I much prefer the microservice model with a formal IPC API.

This converts a simple and solved problem, sharing memory, to a client-server distributed systems problem. There is no need. It's all on the same box.

Re: Dynamic linking

#147

Post by saagarjha » Sat, Jun 27, 2020, 1:48 AM UTC

btilly wrote: post #145

saagarjha wrote: post #123

Earlier quoted context omitted.

> I am firmly of the opinion that content is more important than form. This is quite possibly the stupidest thing I have heard this year. How could you possibly have come up with something that brain-dead? Is this how you normally are, or did you have to reach deep into your mind to find the most inane thing you could possibly say and pollute Hacker News with it? Goodness, I hope so. I don't even want to think about…

This is quite possibly the stupidest thing I have heard this year. How could you possibly have come up with something that brain-dead? That is an opinion. It is your opinion. I happen to have a very different opinion. It may be surprising to you, but surprise doesn't mean that it is trivially wrong. Experience has taught me that if you have good content, then you have something worthwhile to say if I can get past the…

I have read a lot of your writing, much more than you've linked here, simply by being active on this site for just over three years. While I don't really think I could say I know you personally, I think I have at least a little bit of context about you, what your interests are, and what you like to talk about. Your comments have been consistently useful and relevant, and I have no complaints there.

But back to the topic: form and content are not a zero-sum game. You can be eloquent and polite while also conveying useful information, as you mention yourself. You can find people who do this, and it's not a big deal if you're trying to do it yourself. Honestly. If you think about it, the reason I'm replying to you (and why I stick around in some threads and leave others) is both the content and the way it's dicussed: I usually stop interacting when people start hurling insults, because it's just exhausting to deal with it for extended periods of time. It's part of the reason why I occasionally leave Reddit frustrated, why I'm not on Twitter, and why I let other people pull information out of Slashdot and 4chan for me. Really, if I can find one person who says something useful but surrounds it with vicious barbs, and another who can say the same thing but without the acid, why wouldn't I want to go with them? If I can only find the latter, perhaps I'm with the wrong people.

And again, I do personally make some concessions for people I truly do think have exceptional content–but again, these are some concessions. Even Linus–who is usually the poster child of the "who cares how you say it?", who is exceedingly bright and talented, and who is basically a "equal opportunity abuser"–could probably tone it down some. When he is right it's just extra verbiage I need to scroll through and ignore, and when he's wrong (and he is for some things) it just makes him sound extra rude.

Re: Dynamic linking

#148

Post by Shorel » Sat, Jun 27, 2020, 1:56 AM UTC

I think the elephant in the room here is things like Flatpack and Snap, which are basically ugly bastardized versions of static linking.

Re: Dynamic linking

#149

Post by cosmojg » Sat, Jun 27, 2020, 2:01 AM UTC

wahern wrote: post #3

Do any Linux/glibc or Linux/musl systems support static PIE binaries, yet? Without static PIE support you don't benefit from ASLR (at least not fully). This 2018 article seems like a good breakdown of the issues: https://www.leviathansecurity.com/blog/aslr-protection-for-s... OpenBSD has supported static PIE since 2015; not just supported, but all system static binaries (e.g. /bin and /sbin) are built as static PIEs,…

> but the basic takeaway is that dynamic linking in system toolchains and system runtimes is far more mature than static linking.

But that's a chicken-or-egg problem, right? The only way static linking tools will reach parity with dynamic ones in terms of maturity is if static linking replaces dynamic linking as the de facto method of choice.

Re: Dynamic linking

#150

Post by Shorel » Sat, Jun 27, 2020, 2:03 AM UTC

gok wrote: post #67

Why stop there? Package every binary in its own container, too! Most programs only use a few system calls, so you're not really getting anything by sharing a single kernel for the entire system.

Seems to be the general trend. A virtualized kernel per service.