Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sta.li: Static Linux

sta.li

Post reply on HN
Print view View original

81–90 of 99 posts

Re: Sta.li: Static Linux

#81

Post by ChuckMcM »

eekee wrote: post #53
ChuckMcM wrote: post #16

Ah kids, they crack me up! Lot of fun reading that, I looked around briefly but couldn't find my email archive from Sun, but I was in the kernel group when folks got the idea that "Gee if you shared the text segment of libraries, that would give you more memory for your buffer cache or user pages!" One of the guys in my group re-wrote the linker to scan through and realign text segments so that the maximum number of…

that one paragraph... if i were you i'd be wishing i could edit that comment! you most certainly do not "cut down on file load time." the linking process is a complex task, especially if it's to be performed with any efficiency. i remember when starting a large dynamically linked executable was a glacial process on linux, far slower than the time necessary to read the entire executable from disk. no sensible system l…

To be clear, loading data from disk takes milliseconds and re-computing addresses takes nanoseconds. Yes, in less than one tenth the time it takes to read the part of the libraries you have linked into your binary from disk, you can locate and fixup any link references.

This part I don't get "no sensible system loads the entire executable when it's statically linked." If you're a statically linked executable, by definition the entire file is headed into memory, if there was something in the library you didn't use it got edited out in the link step. Now you may mmap the file and fault it in as you go along, but you are going to have the whole thing read.

Re: Sta.li: Static Linux

#82

Post by pjmlp »

ithkuil wrote: post #78
pjmlp wrote: post #29

Earlier quoted context omitted.

> So assuming you are only linking in well-designed libraries This is a very big assumption. Dynamic linkers are also clever enough to only mmap the required parts of dynamic library.

the whole library gets mmapped, there is no point doing otherwise, the mapping itself is cheap. You are probably talking about demand paging (which happens on statically linked binaries.

You probably talking about Linux, there are other types of dynamic loaders out there.

Re: Sta.li: Static Linux

#83

Post by pjmlp »

npsimons wrote: post #72
pjmlp wrote: post #29

Earlier quoted context omitted.

> So assuming you are only linking in well-designed libraries This is a very big assumption. Dynamic linkers are also clever enough to only mmap the required parts of dynamic library.

It's been awhile since I've mucked about with these sorts of things, but I'm glad that my thought about that is confirmed: if static linking only brings in used functions, why doesn't dynamic loading do the same (it does, apparently)? Much of this railing against dynamic loading wasting resources seems like complaining about the wrong things, either bad dynamic linkers, or bad libraries, neither of which will be fixe…

I don't know that well how Linux dynamic loader works, my comment was based on what is possible in other operating systems in general, and what has been done in operating system research.

Re: Sta.li: Static Linux

#84

Post by mpyne »

spc476 wrote: post #28
cgh wrote: post #17

Earlier quoted context omitted.

Static linking doesn't necessarily link the entire library, unless the entire thing compiles to a single .o file. Linkers are smart enough to only link in the object files needed by the program. So assuming you are only linking in well-designed libraries, I guess it's possible that statically-linked software will be smaller since it will leave out the stuff you aren't using.

I recently played around with this, taking a rather small project (around 15,000 lines of code), putting it all into a single file and compiling. It did produce a smaller executable, but the real gain was in making every function static (since it's all in a single file). Doing that, a total of 41 functions were eliminated (either inlined or not used at all). Was it worth the effort? Eh. But it was instructive and I'd…

We did that (as a developer option) with KDE as well, since KDE 2 or thereabouts?

With the automake-based build system you'd pass "--enable-final" and the buildsystem would cat all the source files together and compile the whole damn thing at once (and really stress-test the kernel and gcc).

With KDE 4 I believe it is -DKDE4_ENABLE_FINAL=TRUE passed to cmake.

It was never quite 100%... sometimes you'd run into things like different source files in a modules declaring the same class name, insufficiently-namespaced header include guards, etc. But it was definitely interesting.

Re: Sta.li: Static Linux

#85

Post by mpyne »

npsimons wrote: post #72
pjmlp wrote: post #29

Earlier quoted context omitted.

> So assuming you are only linking in well-designed libraries This is a very big assumption. Dynamic linkers are also clever enough to only mmap the required parts of dynamic library.

It's been awhile since I've mucked about with these sorts of things, but I'm glad that my thought about that is confirmed: if static linking only brings in used functions, why doesn't dynamic loading do the same (it does, apparently)? Much of this railing against dynamic loading wasting resources seems like complaining about the wrong things, either bad dynamic linkers, or bad libraries, neither of which will be fixe…

AFAIK static linking doesn't bring in "used functions".

It brings in used libraries, all at once. E.g. if you used sincos() from math.a, and math.a contained 47 other math functions, then you'd get all 48 math functions in your static binary just from using sincos().

Someone correct me if I'm wrong but I believe it's only with good whole program optimization at link time that it's possible to truly prove that a function is unneeded and exclude it (and then re-link if needed to re-resolve symbols to their new address in virtual memory).

Re: Sta.li: Static Linux

#86

Post by vinkelhake »

ploxiln wrote: post #44
vinkelhake wrote: post #9

On binary sizes: > Linking a stripped hello world program with glibc results in 600kb. Linking it with uclibc in about 7kb. That's nice for uclibc, but we're typically linking dynamically. The comparison should be between dynamically and statically linked binaries. A stripped and dynamically linked hello world results in a 6kb program on my machine (glibc). There's also a lot of handwaving on memory usage in the FAQ.

Yeah - so your stripped and dynamically linked executable is just about as large as a statically built one... and yours still has to link glibc. I can build busybox (a multi-call all-in-one executable, use symlinks to refer to the binary with the name of a tool and it acts like that tool), with init and bourne shell and the minimal set of command-line tools (coreutils remakes and util-linux remakes) into a 600KiB exe…

Busybox is great and it's specifically built to be tiny. I don't really know what the stali guys are going for. Are they building something to compete with busybox or something more general? If it's the latter then a user will probably have a lot of programs that use way more of the standard library than "hello world" does.

As it stands, the FAQ entry is comparing apples and oranges. Comparing full-featured and dynamically linked programs to statically linked, but feature-limited ones is only interesting if you can get by with the feature-limited version. I suspect we're in agreement.

Re: Sta.li: Static Linux

#87

Post by CUViper »

khc wrote: post #8

From the FAQ: "Also a security issue with dynamically linked libraries are executables with the suid flag. A user can easily run dynamic library code using LD_PRELOAD in conjunction with some trivial program like ping. Using a static executable with the suid flag eliminates this problem completely." Have the authors actually tried this? Using LD_PRELOAD with suid programs won't work.

Right - at least with glibc, ld.so unsets most LD_* variables and more for both setuid and setgid programs. Grep for UNSECURE_ENVVARS in glibc source to get the whole list and see how it's used. I'd be very surprised if any other libc implementation didn't do the same.

Re: Sta.li: Static Linux

#88

Post by geminitojanus »

eekee wrote: post #53
ChuckMcM wrote: post #16

Ah kids, they crack me up! Lot of fun reading that, I looked around briefly but couldn't find my email archive from Sun, but I was in the kernel group when folks got the idea that "Gee if you shared the text segment of libraries, that would give you more memory for your buffer cache or user pages!" One of the guys in my group re-wrote the linker to scan through and realign text segments so that the maximum number of…

that one paragraph... if i were you i'd be wishing i could edit that comment! you most certainly do not "cut down on file load time." the linking process is a complex task, especially if it's to be performed with any efficiency. i remember when starting a large dynamically linked executable was a glacial process on linux, far slower than the time necessary to read the entire executable from disk. no sensible system l…

> you most certainly do not "cut down on file load time."

Surely you must be joking eekee.

I know SSDs have made us all forget, but Disks (you know, those spinning piles of rust that most of us still have in computers to permanently store our bits on) are incredibly, painfully slow. Average times for any action on a disk are in milliseconds. That's millions of computational cycles.

I'm sure someone could invent a system where the dynamic linking process is slower than loading a static executable from a disk, but I've yet to find it. I'm also certain that our assumption that dynamic linking is always the way to go will be more and more challenged by the speed of SSDs, which are becoming much closer to RAM in speeds every day. But for today... no way.

Re: Sta.li: Static Linux

#89

Post by geminitojanus »

justincormack wrote: post #63
DougMerritt wrote: post #32

Earlier quoted context omitted.

I remember, but these days the parallel to working set size (depending on the app) is fitting things in cache, or at least cache-awareness -- which some people aim for, but is largely an ignored issue. Page thrashing versus cache thrashing. An issue both back then and today is potentially that dynamic linkage often ends up using an indirection table, unlike static linkage with simple fixups, which then adds at least…

Plus you can inline stuff with static linking.

Whole program optimization is really the killer feature for static linking today. It's possible with dynamic linking, but a static compiler can really go to town shedding weight since lots of libraries have common code structures (see every sufficiently complex C library and its own implementations of various data structures).

Re: Sta.li: Static Linux

#90

Post by Avshalom »

agumonkey wrote: post #67
anon4 wrote: post #37

Earlier quoted context omitted.

I used to run dwm back when I had a machine with just 256MB RAM. The source code is very clean and if you just want to edit some keybindings or the colourscheme, etc. it's so well structured, it's pretty much like editing a config file. It also compiles in no time at all.

I wonder if this was how people felt about C in early days. ASM was the low level frailty, and C the oh-so-clean one-make-away world to create, extend, modify your system.

Lispmachines existed in the early days of C, so no probably not.
Post reply on HN