Live data from Hacker News

An alternative to shared libraries (2008)

kix.in

11–18 of 18 posts

Re: An alternative to shared libraries (2008)

#11
post #6
post #2

I've done mercifully-little C(++) programming in my life, but the very idea of dll's has always seemed extremely fragile to me. Depending that deeply on the system configuration for basic functioning... ylech. I also can't help but wonder if they're obsolete these days, given how incredibly cheap disk storage is? Can anyone who works in that sector speak more to that?

At least one C++ library that I rely on (that produces an incredible number of template instantiations in the object files) ultimately results in a 2 GB shared object file for the debug version (generating DWARF info for 300,000+ objects takes up a lot of space). The test suite for this project compiles and runs about 5,000 executables that link, in some way, to that giant shared object library. Statically linking th…

> Statically linking the test suite would consume on the order of terabytes (I am not sure the exact number; I have never tried) of disk space which just is not feasible for a workstation

I read complaints like this a lot. Have you ever tried using -ffunction-sections -fdata-sections and --gc-sections as in :https://elinux.org/images/2/2d/ELC2010-gc-sections_Denys_Vla...

I find static linking to take up far less disk space when done properly, but I keep seeing comments like this.

Re: An alternative to shared libraries (2008)

#12
As Docker has proved to me, people don't care about saving space anymore. The time of shared libraries has come and gone. And yes I realize you can use shared libraries in Docker, but at that point it's acting more like a static library oddly enough.

Side note - anytime I see Drepper mentioned I cringe a bit. I don't like to badmouth people, but I can't think of a worse figure in recent open source history.

Re: An alternative to shared libraries (2008)

#13
This proposal is crazy town.

If you think about it, if your code is small and clean, you wouldn’t feel the need for shared libraries.

Not so. The key feature of DLLs is to enable software components to evolve independently. When Apple changes the way a button looks in UIKit, all apps get the updated behavior, because UIKit is dynamically linked. This has nothing to do with the app size or its hygeine.

Replacing dylib calls with filesystems and fread/fwrite is absurd. IPC is much harder than library calls. There's the obvious expense of turning every library call into multiple system calls. It's also less flexible. When the interface is a function call, the function can switch implementations from eager to lazy, add or remove caches, etc. But when your interface is fread() you must provide your data up front. Switching a field to computed on-demand is a breaking change.

But the versioning issues are the worst. Reading from Plan9 /dev/mouse returns 49 bytes. I can't add a new field without breaking apps. The equivalent of adding a method in an OO language is now a breaking file format change in this scheme. The post attempts to addresses this:

With filesystems, it’s trivial to add functionality without breaking applications depending on older versions of your FS. That’s because all the compiler sees is a bunch of fopen/fread/fwrites and is not going to complain if the version of the filesystem changes because it doesn’t know.

The app can't break because the compiler can't detect the breakage. Huh?

Alternatively, if you’re thinking of modifying the behavior of your filesystem; consider providing a version file in the root of your FS right from the beginning. Applications would then write the version number they expect to be working with in that file as a way of initializing the filesystem

This sounds very bad:

1. Every API needs to be able to ingest and emit all past versions of themselves, in perpetuity. This is obviously a huge maintenance burden.

2. This does nothing about the reverse problem: how does an app run on multiple versions of the OS? Instead of, say, using reflection to test if a method is present, apps must be prepared to parse the product of every API they use against every OS version they want to support.

3. Applications do not have a single "version number they expect." Apps are built out of multiple components written at different times which consume different versions of the same API. There's no affordance for that.

So we're forced to use some structured data format with named fields, rather than raw bytes. How did this come out of a desire for a "lean, efficient and small" solution?

Re: An alternative to shared libraries (2008)

#14
post #3

Earlier quoted context omitted.

These days I mostly work in JavaScript, and say what you will about NPM, at least it provides a consistent, reliable, central strategy for retrieving and organizing external dependencies (assuming you haven't just built them into your bundle). The same is true of Maven for Java, to some extent. It seems like at least in Linux, the package manager will get your DLL's for you and coordinate all that, but as far as I ca…

I develop small to medium size Windows applications. It's much easier for me to simply put all the DLLs I need in the installer. Just copy everything into the installation directory. Since Windows looks in the immediate directory first, that eliminates most potential conflicts. There are some things you might install onto the system like C++ redistributables, but even recently I have just started copying those DLLs i…

Which totally defeats the purpose of dll's, right? I'm not judging; this just seems like a pretty condemning argument against the practice as a whole

Re: An alternative to shared libraries (2008)

#15
post #14

Earlier quoted context omitted.

I develop small to medium size Windows applications. It's much easier for me to simply put all the DLLs I need in the installer. Just copy everything into the installation directory. Since Windows looks in the immediate directory first, that eliminates most potential conflicts. There are some things you might install onto the system like C++ redistributables, but even recently I have just started copying those DLLs i…

Which totally defeats the purpose of dll's, right? I'm not judging; this just seems like a pretty condemning argument against the practice as a whole

It's not. DLLs also provide a way to share compiled code between programming languages that are not binary compatible.

Re: An alternative to shared libraries (2008)

#16
post #7

So an alternative to shared libraries is RPC over FUSE-like transport? It is a sane idea and may work well, but I think the page overcomplicates it. Why bother with filesystem -- having cryptofs implement "getattr" and "readlink" seems like a total waste. Why not use the things designed for RPC? For example, one may use raw unix sockets (this is how Xorg, and pulseaudio work), or D-Bus (this is how disk mounting in m…

Or just… executables? My shell is "dynamically linked" to every executable on the system. Git is "dynamically linked" to my $PAGER. My IDE is "dynamically linked" to my compiler toolchain (which is itself composed of several pluggable components which are "dynamically linked" to each other).

The low overhead, easily composable, encapsulated and abstracted nature of processes is one of the things that made Unix an unusually functional operating system in the first place. Shared libraries are great, but I think a lot of the time people end up producing and linking a .so when defining an old-fashioned shell-compatible interface would be perfectly good and far less trouble.

Re: An alternative to shared libraries (2008)

#17

This proposal is crazy town. If you think about it, if your code is small and clean, you wouldn’t feel the need for shared libraries. Not so. The key feature of DLLs is to enable software components to evolve independently. When Apple changes the way a button looks in UIKit, all apps get the updated behavior, because UIKit is dynamically linked. This has nothing to do with the app size or its hygeine. Replacing dylib…

Exactly. It’s utopian “freedom” of tyranny like libertarianism. If every app bundled all of its own shared libs, and the FS deduped files and shared library loader deduped memory pages, there would be still be isolation, with sharing AND add the ability to upgrade any piece of software without having to maintain forests of NIX packages to or Habitat individual special libraries to reference count. And, to delete an app, just delete the app, which decrements dedupped files and deletes singular ones... no shared components to clean up! ZFS lz4+dedup plus GNU stow

Re: An alternative to shared libraries (2008)

#18
post #6

Earlier quoted context omitted.

At least one C++ library that I rely on (that produces an incredible number of template instantiations in the object files) ultimately results in a 2 GB shared object file for the debug version (generating DWARF info for 300,000+ objects takes up a lot of space). The test suite for this project compiles and runs about 5,000 executables that link, in some way, to that giant shared object library. Statically linking th…

> Statically linking the test suite would consume on the order of terabytes (I am not sure the exact number; I have never tried) of disk space which just is not feasible for a workstation I read complaints like this a lot. Have you ever tried using -ffunction-sections -fdata-sections and --gc-sections as in : https://elinux.org/images/2/2d/ELC2010-gc-sections_Denys_Vla... I find static linking to take up far less dis…

MSVC linker also supports removing unused functions/data from the output binary: https://msdn.microsoft.com/en-us/library/bxwfs976.aspx. These can be applied irrespective of compiler optimizations, so you should still be getting good PGD debug symbols.

Haven't tried it personally, but it's worth a try.

Post reply on HN