Live data from Hacker News

Static Linux

sta.li

41–50 of 61 posts

Re: Static Linux

#41
post #30

Earlier quoted context omitted.

That's the party line. It's often wrong. If two copies of the same program are running, they share memory for code. For a shared library to reduce memory consumption, there must be multiple different programs using the same version of the same library. That's not all that common, beyond very basic libraries such as "libc". Linking to a shared library brings in and initializes the whole library, even if you only need…

That's not all that common Isn't it? Usually distros target their packages to a single library version, and often people run suites (Gnome, KDE, etc) that use a similar set of libraries in their different processes.

Indeed. ldd any substantial GTK app and scroll past the dependencies. They are huge. Most of them are shared across applications.

Desktop would be crippled if every app was compiled with the whole stack of X, toolkit and Gnome libraries linked in statically.

Re: Static Linux

#42

Cool idea. Not enough people seem aware of executable packers like UPX http://upx.sourceforge.net/ though. These are excellent tools to keep the size down when using large static binaries. By compressing the file on disk and decompressing in memory you often wind up with a smaller and sometimes even faster loading (depending on disk IO speed vs decompression speed) package. I got a static Qt binary from 4 MB down to…

I found a bug with an offline documentation reader (can't remember which) that if you pack one of the QT components (qwindows.dll) the application wouldn't load anymore. Still not sure whether to report a bug and who to report it to. Yes UPX is cool, but I don't think it's 100% compatible.

You can usually set UPX options to fix that kind of thing. You may have stripped necessary linking information. I usually use it on statically linked things though, so no issue there.

Re: Static Linux

#43

Cool idea. Not enough people seem aware of executable packers like UPX http://upx.sourceforge.net/ though. These are excellent tools to keep the size down when using large static binaries. By compressing the file on disk and decompressing in memory you often wind up with a smaller and sometimes even faster loading (depending on disk IO speed vs decompression speed) package. I got a static Qt binary from 4 MB down to…

Ah, and pklite pro and lzexe from the olden days for winDOwS shit once upon a time.

Though the goal should be generating less code. Link in fewer dependencies, reduce features, DRY up duplicate logic and cut LoC. Also compile with -DNDEBUG -O2 -g- and whatever LTO switches are available for whole program optimization if you're statically linking everything together. Also be sure to include static dependencies of other static dependencies like zlib (-lz), or you'll inevitably end up with missing symbol errors when compiling a final program. LTO cuts out all (most) of the shit that you don't need and attempts to optimize across translation units.

Furthermore a consideration against static linking, on most platforms, if the same shared library is already loaded, it's reused by mmaping it into a process. Not sure that duplicating code is going to reduce memory usage or the IO it takes to load from disk. Giant runtimes like Go, Ruby, Python and fucking Java shouldn't be duplicated N times... That's just wasteful. (I hate any language with an epic runtime or VM that includes the world to do anything.). Libraries should be reserved for the few redundant things that take tons of code to implement and change very little.

If anyone wants to compile a Linux system from scratch, try LFS and hackaround with static linking. It may take patches, extra flags to get what you want.

Hope Static Linux scales, because it's easier to upgrade static programs without dependency hell but the increased memory usage of duplicated code might not be so great of a tradeoff.

Another hack would be to statically compile every system program in each directory together (/sbin/, /bin/, and parts of /usr/bin, etc) into a single executable per directory that is then symlinked to itself to select which "program" to load via argv[0]. It will be one giant exe per directory, but it will be cached basically all the time and with LTO, there won't be much duplication as with N programs compiled separately. This would take a main which dispatches to other renamed original mains and renaming all symbol conflicts across all translation units.

    /bin/static
    /bin/[ -> /bin/static
    /bin/false -> /bin/static
    /bin/true -> /bin/static
(Probably want to use hard links also)

Re: Static Linux

#44

I wish Linux would replace dynamic libraries (especially ones referencing specific paths) with a system based on the library's hash. Then we could have a single lib folder with 1 copy of each library, and get ourselves out of dependency hell by just making dynamic loading act like static loading. We could even download libraries from the web on the fly, if needed. Heck it would even remove a lot of the need to compil…

> I wish Linux would replace dynamic libraries (especially ones referencing specific paths) with a system based on the library's hash. Given this scheme, how would you distribute a security patch? Is each user of the library supposed to re-compile against the patched library? Also, a program A depends on library B and library C v1.1. Library B also depends on C, but v1.2. Which gets used? > More often than not, when…

Distro packagers generate a listing of public symbols (tagging a symbol with the version number when it is introduced) to catch incompatibilities. It's like static typing, it takes a bit of getting used to but it's very reliable.

https://www.debian.org/doc/manuals/maint-guide/advanced.en.h...

Re: Static Linux

#45
"Of course Ulrich Drepper thinks that dynamic linking is great, but clearly that’s because of his lack of experience and his delusions of grandeur." I find these kind of comments reflect veerryyy poorly on their authors..

Re: Static Linux

#46

I wish Linux would replace dynamic libraries (especially ones referencing specific paths) with a system based on the library's hash. Then we could have a single lib folder with 1 copy of each library, and get ourselves out of dependency hell by just making dynamic loading act like static loading. We could even download libraries from the web on the fly, if needed. Heck it would even remove a lot of the need to compil…

> The notion of being able to fix an app by merely upgrading a library it depends on has not worked out in practice. It works in practice if developers and maintainers adhere to semantic versioning. Unfortunately, there are numerous packages that don't adhere to this standard and that's when widespread breakage occurs.

An example of where this has worked (though admittedly the reason it has continued to work as long as it has is that the size of the community has dwindled down to a very small, manageable number over the last 20 years) is AmigaOS, where there are 30 year old libraries that are still updated, and where the updates are still expected to be drop-in replacements for the previous version.

Re: Static Linux

#47
post #22

> Because dwm is customized through editing its source code, it’s > pointless to make binary packages of it. This keeps its userbase > small and elitist. No novices asking stupid questions. I never understood why the authors of dwm thought this was a "nice" feature of configuration via source code.

Consider that their goal is not to have a huge community, but to have a piece of software that does what they want, and that dealing with a community is a lot of work. So they've chosen to use this as a filter in order to exclude people they perceive as causing more trouble than it's worth, and limit their community to people who pass through their "trial by fire" of having to be both willing and able to deal with compiling from source.

Re: Static Linux

#48

I wish Linux would replace dynamic libraries (especially ones referencing specific paths) with a system based on the library's hash. Then we could have a single lib folder with 1 copy of each library, and get ourselves out of dependency hell by just making dynamic loading act like static loading. We could even download libraries from the web on the fly, if needed. Heck it would even remove a lot of the need to compil…

> The notion of being able to fix an app by merely upgrading a library it depends on has not worked out in practice. It works in practice if developers and maintainers adhere to semantic versioning. Unfortunately, there are numerous packages that don't adhere to this standard and that's when widespread breakage occurs.

And if developers would just write bug-free code then also life would be much simpler. Sadly neither of these things is going to happen, so we need to deal with it somehow.

Re: Static Linux

#49

Are there any static BSDs?

You can build NetBSD to be statically linked [1]. It is not the default, but it is trivial to do, one line config change. Note that you can build NetBSD from source on any system, eg Linux or OSX and then boot it in a VM, so you dont really need a binary "distro".

[1] https://www.netbsd.org/docs/guide/en/chap-build.html#chap-bu...

Re: Static Linux

#50
post #8

Earlier quoted context omitted.

A similarly interesting project is the musl-based Sabotage Linux with its own tiny but concurrent package manager: https://github.com/sabotage-linux/sabotage

Sabotage is such a wonderful idea and I really hope it succeeds. It's not nearly usable right now, though.

What do you have a problem with?
Post reply on HN