Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

32-bit x86 Position Independent Code – It's That Bad

ewontfix.com

Post reply on HN
Print view View original

21–30 of 104 posts

Re: 32-bit x86 Position Independent Code – It's That Bad

#21

Post by pilif »

deweerdt wrote: post #9
bandrami wrote: post #2

The more I play around with musl (the author's C library) the more I'm convinced dynamic linking was not worth the trouble.

One of the main reasons for dynamic linking has become irrelevant, I believe: the availability of disk and memory space has grown faster than the size of the binary objects.

IMHO, static linking, be it using a mechanism alike what npm is doing or be it plain-old static linking of binaries also means "you link it, you own it".

For every package, you link statically, you as the parent package owner become responsible for all security flaws of all the packages you link statically.

And by "responsible" I mean: Every dependent security announcement of a dependent package also becomes your security announcement.

If you're AWESOMY 1.1 and you link statically against openssl and openssl announces a security flaw, then you better and quickly release AWESOMY 1.1.1 with an accompanying security announcement too.

Are you willing to do this? Do you trust the chain of dependencies all the way down to also be willing to do the same?

As a responsible developer, I'd much rather delegate that responsibility away to a packager or even the user, especially with well-known libraries like openssl.

If I'm owning AWESOMY 1.1 and link dynamically against openssl, then I don't have to do anything when openssl releases a security announcement. I can, if I want to, inform my users to maybe update openssl, but with some likelihood they are already doing this anyways for some other package.

For me as a developer, this is considerably more convenient.

Yes. Static linking has huge advantages for me as a developer too, but it also comes with a great many additional responsibilities I'm personally not willing to take on, also because I don't trust my dependencies to be as diligent about their dependencies.

Re: 32-bit x86 Position Independent Code – It's That Bad

#22

Post by deathanatos »

yuriks wrote: post #19
bandrami wrote: post #8

Earlier quoted context omitted.

Oddly enough I was thinking about openssl updates specifically. Did you remember to restart all your long-running clients ? (I'm a sysadmin who occasionally gets roped into doing development...) Dynamic linking makes security more difficult to reason about, which is a Bad Thing. It has many pluses, but also many minuses. And with symbol versioning it gets very problematic to actually figure out what your real code pa…

And when you update all your clients with static linked OpenSSL, did you also remember to restart every single one of them? These seem like equivalent problems to me.

Not quite, I think. With a .so, I can ask `lsof` which services on the machine require restart: the ones that haven't are linked to a deleted so.

And you wouldn't need to restart the clients with static OpenSSL — you need to recompile them. And with static linking, I'm not sure how you would easily determine the linked version out to say, the minor or the micro. (Perhaps, if this is your OS's thing like nix, the package manager keeps track…)

Re: 32-bit x86 Position Independent Code – It's That Bad

#23

Post by deweerdt »

pilif wrote: post #21
deweerdt wrote: post #9

Earlier quoted context omitted.

One of the main reasons for dynamic linking has become irrelevant, I believe: the availability of disk and memory space has grown faster than the size of the binary objects.

IMHO, static linking, be it using a mechanism alike what npm is doing or be it plain-old static linking of binaries also means "you link it, you own it". For every package, you link statically, you as the parent package owner become responsible for all security flaws of all the packages you link statically. And by "responsible" I mean: Every dependent security announcement of a dependent package also becomes your sec…

I believe that what you describe is the remaining argument in favor of dynamic linking.

That said, I'd like to point out that it is convenient for the developer - since customers are ultimately interested to know if your software is vulnerable, and you'll have to explain how the vulnerability affects it -.

It's also a double edged sword: openssl has a good ascending compatibility record, but that can't be said about all user space libraries. IOW, it's tricky to guarantee that your software will work flawlessly across all the incarnations of CentOS 6.x, if you have a lot of external dependencies, for example.

Re: 32-bit x86 Position Independent Code – It's That Bad

#24

Post by zvrba »

deweerdt wrote: post #9
bandrami wrote: post #2

The more I play around with musl (the author's C library) the more I'm convinced dynamic linking was not worth the trouble.

One of the main reasons for dynamic linking has become irrelevant, I believe: the availability of disk and memory space has grown faster than the size of the binary objects.

This is an oft-repeated mantra, but is it really true? I don't have a linux desktop running here, but try an experiment: start up a desktop environment such as KDE, and write "free -m" and see how much is there under "shared" heading. Without shared libraries, some extra memory corresponding to some multiple of that number would be used.

To calculate exactly how much memory is saved by shared libraries, you'd need to write a kernel module to walk the internal structures describing physical pages and summing reference counts of used pages. Maybe it's already been done?

Re: 32-bit x86 Position Independent Code – It's That Bad

#25

Post by microtonal »

deathanatos wrote: post #22
yuriks wrote: post #19

Earlier quoted context omitted.

And when you update all your clients with static linked OpenSSL, did you also remember to restart every single one of them? These seem like equivalent problems to me.

Not quite, I think. With a .so, I can ask `lsof` which services on the machine require restart: the ones that haven't are linked to a deleted so. And you wouldn't need to restart the clients with static OpenSSL — you need to recompile them . And with static linking, I'm not sure how you would easily determine the linked version out to say, the minor or the micro. (Perhaps, if this is your OS's thing like nix, the pac…

Tip for Debian/Ubuntu users: checkrestart from the debian-goodies package is a nice wrapper around lsof that lists running binaries that rely on outdated solibs:

http://manpages.debian.org/cgi-bin/man.cgi?query=checkrestar...

https://gehrcke.de/2014/06/good-to-know-checkrestart-from-de...

It not only shows processes that run older solibs, but in the case of services, it will also give you the commands to restart them :).

Re: 32-bit x86 Position Independent Code – It's That Bad

#26

Post by microtonal »

deweerdt wrote: post #23
pilif wrote: post #21

Earlier quoted context omitted.

IMHO, static linking, be it using a mechanism alike what npm is doing or be it plain-old static linking of binaries also means "you link it, you own it". For every package, you link statically, you as the parent package owner become responsible for all security flaws of all the packages you link statically. And by "responsible" I mean: Every dependent security announcement of a dependent package also becomes your sec…

I believe that what you describe is the remaining argument in favor of dynamic linking. That said, I'd like to point out that it is convenient for the developer - since customers are ultimately interested to know if your software is vulnerable, and you'll have to explain how the vulnerability affects it -. It's also a double edged sword: openssl has a good ascending compatibility record, but that can't be said about…

IOW, it's tricky to guarantee that your software will work flawlessly across all the incarnations of CentOS 6.x,

It's not so tricky, since Red Hat specifies nowadays what guarantees can be expected for what packages:

https://access.redhat.com/articles/rhel-abi-compatibility#Ap...

Re: 32-bit x86 Position Independent Code – It's That Bad

#27

Post by leaveyou »

cma wrote: post #18
gwu78 wrote: post #13

There was a time when shared libraries were a necessity. Not to mention supporting myriad architectures. That was a different era. Conditions have changed. Of course, the designs and implementations have not. They are still in heavy use, 20 years later. What "just works" is never questioned. I am glad to see some people questioning the old assumptions. Unrelated question: Why does Linux pass arguments in registers in…

registers are much faster than stack, assuming you immediately make use of them; to use them from the stack you have to load them into registers anyway in most cases

In theory passing args in registers is faster because you avoid one copy in simple/small routines (the ones that perform a very simple op on the reg arguments and return). In practice there are way more complex routines than simple routines and the arg from register is copied back on the stack in the local variables area of the routine because it needs that register to perform some op or simply because it needs that reg to pass an argument to a subroutine. So in practice, the arg copy is only shifted from the caller to the callee.

Re: 32-bit x86 Position Independent Code – It's That Bad

#28

Post by bandrami »

pilif wrote: post #21
deweerdt wrote: post #9

Earlier quoted context omitted.

One of the main reasons for dynamic linking has become irrelevant, I believe: the availability of disk and memory space has grown faster than the size of the binary objects.

IMHO, static linking, be it using a mechanism alike what npm is doing or be it plain-old static linking of binaries also means "you link it, you own it". For every package, you link statically, you as the parent package owner become responsible for all security flaws of all the packages you link statically. And by "responsible" I mean: Every dependent security announcement of a dependent package also becomes your sec…

For every package, you link statically, you as the parent package owner become responsible for all security flaws of all the packages you link statically

Well, speaking as an administrator rather than a developer: a developer shouldn't be making that kind of policy decision (time of symbol resolution) to begin with barring a strong technical need (plug-in based architecture, etc.).

I mean, 90% of packages don't care whether their libraries are dynamic or static, but the ones that do can still be annoying. (Coreutils, of all things, requires dynamic linkage for stdbuf, though that seems to be considered a bug by the maintainer; on the other end of the spectrum, getting Perl to even build statically is like pulling teeth, and that was a design decision.)

Do you trust the chain of dependencies all the way down to also be willing to do the same?

The only time that is a problem is those cases where the package developer literally plunks down a bunch of code from his upstream into his own tree (think the embedded glib inside pkg-config, or the hideous monstrosity that is gnulib). I think all sane people agree this is bad -- if you aren't significantly altering the code (at which point it's "yours") there's no sense in doing that and risking a sync problem. But that's not even static linkage; that's just literal code-sharing.

Re: 32-bit x86 Position Independent Code – It's That Bad

#29

Post by pjc50 »

deweerdt wrote: post #23
pilif wrote: post #21

Earlier quoted context omitted.

IMHO, static linking, be it using a mechanism alike what npm is doing or be it plain-old static linking of binaries also means "you link it, you own it". For every package, you link statically, you as the parent package owner become responsible for all security flaws of all the packages you link statically. And by "responsible" I mean: Every dependent security announcement of a dependent package also becomes your sec…

I believe that what you describe is the remaining argument in favor of dynamic linking. That said, I'd like to point out that it is convenient for the developer - since customers are ultimately interested to know if your software is vulnerable, and you'll have to explain how the vulnerability affects it -. It's also a double edged sword: openssl has a good ascending compatibility record, but that can't be said about…

Dynamic linking is required for anything resembling a plugin architecture (such as PAM).

Re: 32-bit x86 Position Independent Code – It's That Bad

#30

Post by bandrami »

zvrba wrote: post #24
deweerdt wrote: post #9

Earlier quoted context omitted.

One of the main reasons for dynamic linking has become irrelevant, I believe: the availability of disk and memory space has grown faster than the size of the binary objects.

This is an oft-repeated mantra, but is it really true? I don't have a linux desktop running here, but try an experiment: start up a desktop environment such as KDE, and write "free -m" and see how much is there under "shared" heading. Without shared libraries, some extra memory corresponding to some multiple of that number would be used. To calculate exactly how much memory is saved by shared libraries, you'd need to…

some extra memory corresponding to some multiple of that number would be used

Depends. If I have 7 instances of my terminal emulator loaded (say I hadn't discovered tmux yet or something), the loader can share their .rodata and .text segments, and in many cases it does (YMMV; heuristics apply; void where prohibited; etc.). So a lot of things that are in shared libraries right now might still be only loaded into memory once if their binaries are segmented correctly.

The Plan9 people (Plan9 doesn't do dynamic linking) claim that the memory savings they get from skipping the relocation overhead are greater than the memory hit from the times that the same stuff does get loaded multiple times, though obviously always take self-promotion with a grain of salt.

Use case probably matters -- my servers run few processes to begin with, and it's often a lot of versions of the same process, whereas my laptop runs a ton of very different processes. Same sort of argument that makes me happy with udev on my laptop while also very happy with a static /dev tree on my servers.

Post reply on HN