Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Dynamic linking

drewdevault.com

Post reply on HN
Print view View original

191–200 of 249 posts

Re: Dynamic linking

#191

Post by Aissen »

tsimionescu wrote: post #183

> The total download cost to upgrade all binaries on my system which were affected by CVEs in 2019 is 3.8 GiB. This is reduced to 1.0 GiB if you eliminate glibc. The upgrade problem has almost nothing to do with download size. The real problem is that you have > 100 binaries which depend on those libraries, and instead of having the library authors go and update the library, you need each team responsible for one or…

> instead of having the library authors go and update the library, you need each team responsible for one or more binaries to go and take the new library and release a new version of their binary.

Wait, so you're telling me dynamic libraries are a way for distros to ship their org chart?

Maybe it's time in 2020 for automated and deterministic builds to allow anyone to ship a library update and have dependent packages being rebuilt automatically if needed.

Re: Dynamic linking

#192

Post by tannhaeuser »

setr wrote: post #24

I believe the original reasoning for Dynamic Linking wasn't performance gains, but security gains -- someone described the driving story to me as essentially a found vulnerability in a very common library required updating and re-compiling everything on every system , scarring sysadmins globally and permanently; the space saving and performance aspects came up as later "bonuses". I have little memory of the details o…

Don't forget that the principles of the LGPL (originally the "library GPL") were/are pretty much tied to the technical concept of a shared library object that an end-user can change, etc; IANAL so I can't say anything about the legal soundness of this interpretation.

Moreover, in a recent discussion about this topic on HN, someone said the introduction of shared libs into the Linux user space was mainly in support of porting X Windows to Linux (supposedly because of binary video drivers or to accomodate MIT-licensed code?), but I haven't found any supporting reference for that.

glibc's maintainer Ulrich Drepper also has pretty strong opinions on static linking [1]; no matter what you think about this technically, or Ulrich personally, his paper "How to write shared libraries" [2] is considered reference material on the subject.

Personally, I think that the over-use of shared libs in the Linux userland clearly serves no purpose if users flock to entire new layers of abstractions (eg Docker-like containers) to isolate their app delivery from the IMHO overengineered mechanisms in ELF and ld.so with their multiple RUNPATHs, configs, loader scripts, and versioned glibc symbols (on top of build-time libtool/autootols) that still doesn't seem to get to the point. While the LSB effort for more uniform Linux distros isn't dead, it doesn't seem to be taken seriously. Idk, but maybe the GNU folks also see the lack of binary compat for Linux apps as a desideratum, to frustrate any and all attempts to ship binary apps?

[1]: https://web.archive.org/web/20100527213559/http://people.red...

[2]: https://akkadia.org/drepper/dsohowto.pdf

Re: Dynamic linking

#193

Post by eeereerews »

Shorel wrote: post #148

I think the elephant in the room here is things like Flatpack and Snap, which are basically ugly bastardized versions of static linking.

Ugly maybe, but they're do kinda combine the advantages of both. They have the portability and reproducibility advantages of static linking, while still letting you take them apart and change the dependencies like with dynamic linking if you really want to.

Re: Dynamic linking

#194

Post by microtonal »

tsimionescu wrote: post #187
microtonal wrote: post #185

Earlier quoted context omitted.

The real problem is that you have > 100 binaries which depend on those libraries, and instead of having the library authors go and update the library, you need each team responsible for one or more binaries to go and take the new library and release a new version of their binary. This is a solvable problem. Package managers such as Nix and Guix rebuild packages if any of their transitive dependencies have been change…

I was talking about finding vulnerabilities on end-user systems and servers. I don't know about others, but I don't generally keep compiler tool chains for C, C++, Java, Go and maybe 1-2 others on my systems and on my servers in case I need to rebuild all of my binaries.

I was talking about finding vulnerabilities on end-user systems and servers. I don't know about others, but I don't generally keep compiler tool chains [...]

You don't have to, most NixOS/Guix systems use binary caches. So, their build clusters do the work for the packages included the nixpkgs/guix package sets. If your organization builds their own packages in top of that, you can use a CI plus your private cache (or something like Cachix).

Re: Dynamic linking

#195

Post by spc476 »

saagarjha wrote: post #154
wahern wrote: post #131

Earlier quoted context omitted.

But where in its own binary? To support ASLR of code (a ROP mitigation), especially fine-grain randomization, function call sites can't use static addressing, either absolute or relative, pointing directly to a function. To support ASLR of code you can either rewrite every function call site on load (kinda similar to DLLs on Windows), or use one or more tables that are updated at runtime (PLTs, GOT, etc). BSD and Lin…

I know how ASLR/ROP/PLT/GOT/RELRO work, but I'm still not understanding what's going on here. When you statically link a binary, you get…one file. And that gets loaded into memory together, and to make it PIE you have all the jumps be pc-relative. Like, a straight up jump instruction to a fixed offset. So where the the room for a table like this? (Unrelated, but since you brought it up: kbind is IMO not a very good m…

When I participated in NaNoGenMo 2015 [1], I ended up having to emulate a very small subset of MS-DOS [2]. That meant loading in an MS-DOS `.EXE` file into memory and running it. I had a 32-bit Linux system, so I was able to use the `vm86()` system call to run the 16-bit code natively. The `.EXE` file had basically two sections---the first the actual binary code (and data and what have you). This was followed by another section that contained offsets into the code that was loaded that needed to be updated with the proper address, and it took all of five lines of code to implement [3]. Once used, that segment can be discarded. You don't need position independent code to implement address space layout randomization (it makes it easier), you just need a table to rework the absolute addresses (downside---it may take some time).

[1] National Novel Generation Month

[2] https://github.com/spc476/NaNoGenMo-2015, specifically, https://github.com/spc476/NaNoGenMo-2015/blob/master/C/msdos...

[3] lines 337 to 343 of `msdos.c` [2]

Re: Dynamic linking

#196

Post by saagarjha »

ndesaulniers wrote: post #172
Vogtinator wrote: post #4

> On average, dynamically linked executables use only 4.6% of the symbols on offer from their dependencies. That's correct, but also very misleading and leads to the wrong conclusion. The dynamically linked library has references to itself, externally visible or not. It would be wrong to claim that Application.run(); only uses a single symbol of a library. > A good linker will remove unused symbols. With LTO or -f{fu…

> On average, dynamically linked executables use only 4.6% of the symbols on offer from their dependencies. For more on this, I highly recommend these two posts, which show how modern symbol tables optimize for symbols NOT being found within a given shared object via the use of bloom filters. 1. https://flapenguin.me/elf-dt-hash 2. https://flapenguin.me/elf-dt-gnu-hash

This was a great look at something that in hindsight understand must exist but was totally unaware of. Thanks!

Re: Dynamic linking

#197

Post by saagarjha »

spc476 wrote: post #195
saagarjha wrote: post #154

Earlier quoted context omitted.

I know how ASLR/ROP/PLT/GOT/RELRO work, but I'm still not understanding what's going on here. When you statically link a binary, you get…one file. And that gets loaded into memory together, and to make it PIE you have all the jumps be pc-relative. Like, a straight up jump instruction to a fixed offset. So where the the room for a table like this? (Unrelated, but since you brought it up: kbind is IMO not a very good m…

When I participated in NaNoGenMo 2015 [1], I ended up having to emulate a very small subset of MS-DOS [2]. That meant loading in an MS-DOS `.EXE` file into memory and running it. I had a 32-bit Linux system, so I was able to use the `vm86()` system call to run the 16-bit code natively. The `.EXE` file had basically two sections---the first the actual binary code (and data and what have you). This was followed by anot…

This isn’t my area of expertise, but I think on Windows relocations are actually handled quite similarly (as a table of locations that the loader uses to patch with).

Re: Dynamic linking

#198

Post by wahern »

saagarjha wrote: post #154
wahern wrote: post #131

Earlier quoted context omitted.

But where in its own binary? To support ASLR of code (a ROP mitigation), especially fine-grain randomization, function call sites can't use static addressing, either absolute or relative, pointing directly to a function. To support ASLR of code you can either rewrite every function call site on load (kinda similar to DLLs on Windows), or use one or more tables that are updated at runtime (PLTs, GOT, etc). BSD and Lin…

I know how ASLR/ROP/PLT/GOT/RELRO work, but I'm still not understanding what's going on here. When you statically link a binary, you get…one file. And that gets loaded into memory together, and to make it PIE you have all the jumps be pc-relative. Like, a straight up jump instruction to a fixed offset. So where the the room for a table like this? (Unrelated, but since you brought it up: kbind is IMO not a very good m…

Turns out I was the confused one, not you. I thought it was at least possible to generate binaries with multiple segments, each with their own PLT, GOT, and .text sections, similar to putting .text and .data sections in separate segments. That way the relative position of all functions wouldn't be static. But it seems there's no way to generate binaries like this using the standard tools.

Re: Dynamic linking

#199

Post by saagarjha »

ridiculous_fish wrote: post #167

This is a crazy conversation! Dynamic linking allows the system to decide the UI. Static linking means that the UI cannot evolve. Imagine statically linking UIKit or Android's UI library!

Yeah but then you have CFLinkedOnOrAfter to negate that ;)

Re: Dynamic linking

#200

Post by saagarjha »

cryptonector wrote: post #169
cosmojg wrote: post #164

Earlier quoted context omitted.

So let's fix the problem! How can we get better semantics? Have any hypothetical solutions in mind?

I've outlined it already, but let me do it again for you: 1) when making a static link archive (a .a file), include a .o with a static symbol(s) whose values(s) records the metadata that ELF would have recorded, which here is: a) dependencies, b) how to find them, c) the mapfile / version-script (at least which symbols are symbolic, protected, or demoted to local, and which, if any, are interposers. 2) on final link-…

Is it literally called libstool…
Post reply on HN