Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Oasis: a small statically-linked Linux system

github.com

Post reply on HN
Print view View original

71–80 of 169 posts

Re: Oasis: a small statically-linked Linux system

#71

Post by nybble41 »

Subsentient wrote: post #12
HALtheWise wrote: post #7

This seems to "statically link" binaries in the sense that each program it ships with is a standalone binary. Does anyone know whether it's possible to truly statically link an entire Linux install, in the sense that the whole system is a single statically linked file including the kernel, display manager, web browser, etc so that link-time optimization can deduplicate code across the entire system?

Yes, I know that it's NOT possible. Anything that uses the same libraries, e.g. GTK+, will end up calling gtk_init() multiple times. Callback functions with the same names will collide at link-time. Despite the madness of the code that would be generated if the compiler allowed it, since it doesn't, you can expect a list of linker errors so long, that you can go to bed and wake up in the morning and it's still spewin…

It could be done. Obviously you would only run one of the main functions (via a wrapper) per process invocation and you would need to rename conflicting global symbols at some point before the link step, but this is basically how tools like busybox work. If the application uses global constructors or init functions then those would require special handling since they bypass main—C tends to work better than C++ in this regard. Applying LTO to the resulting code for inter-application optimization would be an interesting twist.

Re: Oasis: a small statically-linked Linux system

#72

Post by Subsentient »

AnIdiotOnTheNet wrote: post #21
Subsentient wrote: post #13

Honestly, I'm team dynamic linking. I prefer to have things clearly separated in functionality and easily upgradeable. Statically linking all the OS utilities to their dependency libraries, over and over again? Dear god that sounds awful.

There's no reason that there needs to be such extremes as either all statically linked or all dynamically linked. History has proven that dynamic linking causes a large number of headaches, and static linking has drawbacks as well. There is a reasonable middle ground: base system functionality that is utilized by most software (stdlib, cryptography, networking, gui, etc) should be dynamic and everything else should b…

>>There is a reasonable middle ground: base system functionality that is utilized by most software (stdlib, cryptography, networking, gui, etc) should be dynamic and everything else should be static

I disagree, I tend to want the opposite. I want big, bloated libraries like Qt or Gtk or Boost to be dynamically linked, and OS facilities like libc to be statically linked.

Re: Oasis: a small statically-linked Linux system

#73

Post by tetraca »

siraben wrote: post #55

Does statically linking everything not lead to much higher disk usage, espcially when you have thousands of binaries? Drew Devault has an analysis[0] that appears to claim otherwise. [0] https://drewdevault.com/dynlib.html

It might have been a concern long ago when space was a premium, but these days in the world of >=1TB drives you could probably build your entire software library in the most horrifying, space inefficient way, and it would still be completely and utterly dwarfed by the user's media library. Excepting games, I would bet most anyone's software library would likely comfortably fit under 128 GB. The photos, games, music, and video would be what necessitates more. And I suspect even then, a lot of people don't bother to store these things locally besides the games and photos.

Re: Oasis: a small statically-linked Linux system

#74

Post by kevin_thibedeau »

gameswithgo wrote: post #15
mittermayr wrote: post #8

Would be cool to get a bit of a "why this matters" intro on repos like this. I clicked through the details, but left wondering if I have any potential use for this. Can I compile this onto a USB stick and use it as a throw-away boot Linux for maintenance tasks? Can I cross-compile this for embedded devices? What is the statically-linked advantage here? Not trying to minimize the effort, I would actually love to see m…

As someone who uses linux casually, I often experience pain when I download a program, try to run it, and it fails because some dependency is not installed or is not the right version. So you try to figure out how to install it and it in turn needs a couple of things. All of this works fine if whatever package manager your distro uses has the program you want, but the package managers don't have evertying! So I'd lov…

If you need exotic libraries, you install them into /usr/local/lib with GNU Stow for secondary package management. Then point your loader there first when starting programs that depend on them. Your distro stays pristine and the program runs so everything is good.

Re: Oasis: a small statically-linked Linux system

#75

Post by 1996 »

AnIdiotOnTheNet wrote: post #56
1996 wrote: post #49

Earlier quoted context omitted.

I would be surprised if there was not a way to take a dynamic executable, "add in" all the required libraries, and return a fat static executable

If you know of such a utility I'd like to see it, because I've looked and never found it. The closest I've seen are things like AppImage.

Shell scripting and ld tools should allow you to make that yourself.

EDIT: people are apparently entitled to solutions for their problems...

> Care to elaborate?

> Show us a script then! Your internet fame awaits.

I'm not interested in fame, or in reinventing the wheel.

Apparently, you are unfamiliar with this process, so TLDR: there are basically 2 approaches, in order of complexity: 1. working with ldtools or 2. running the executable to find the various parts then dumping a binary, (3 if you consider putting the libraries in a directory then run with LD_LIBRARY_PATH=/thisdir myexecutable)

Let me google some existing tools for you:

https://github.com/systems-nuts/dynamic_to_static

http://bitwagon.com/jumpstart/jumpstart.html

http://statifier.sourceforge.net/

https://www.ucc.asn.au/~dagobah/things/make-static.html

https://web.archive.org/web/20130114222319if_/http://dagobah... with patch and explanation on https://stackoverflow.com/questions/17390722/how-can-i-conve...

A better approach from 15 years ago is slinky, so that you can "update" the libraries if needed: https://www.usenix.org/legacy/publications/library/proceedin...

There's nothing fancy in there. It just requires some effort.

Re: Oasis: a small statically-linked Linux system

#76

Post by mforney »

fmakunbound wrote: post #35

Ok maybe this is dumb question and is addressed somewhere: If a security problem is found, e.g. in muscl (the C lib), then is the user supposed to rebuild everything that statically linked it??

Yes, but with the oasis build system this is just a single command to do an incremental build that relinks your binaries and takes a matter of seconds.

The user experience is essentially the same as updating your system on a dynamically-linked system. There is no "partial build" where some binaries get relinked but not others.

Re: Oasis: a small statically-linked Linux system

#77

Post by AnIdiotOnTheNet »

1996 wrote: post #75
AnIdiotOnTheNet wrote: post #56

Earlier quoted context omitted.

If you know of such a utility I'd like to see it, because I've looked and never found it. The closest I've seen are things like AppImage.

Shell scripting and ld tools should allow you to make that yourself. EDIT: people are apparently entitled to solutions for their problems... > Care to elaborate? > Show us a script then! Your internet fame awaits. I'm not interested in fame, or in reinventing the wheel. Apparently, you are unfamiliar with this process, so TLDR: there are basically 2 approaches, in order of complexity: 1. working with ldtools or 2. ru…

Care to elaborate? Because I'm not aware of any tool that could be used to combine a binary and all its .so dependencies (and all of theirs recursively) into a single statically-linked binary.

Re: Oasis: a small statically-linked Linux system

#78

Post by AnIdiotOnTheNet »

Subsentient wrote: post #72
AnIdiotOnTheNet wrote: post #21

Earlier quoted context omitted.

There's no reason that there needs to be such extremes as either all statically linked or all dynamically linked. History has proven that dynamic linking causes a large number of headaches, and static linking has drawbacks as well. There is a reasonable middle ground: base system functionality that is utilized by most software (stdlib, cryptography, networking, gui, etc) should be dynamic and everything else should b…

>>There is a reasonable middle ground: base system functionality that is utilized by most software (stdlib, cryptography, networking, gui, etc) should be dynamic and everything else should be static I disagree, I tend to want the opposite. I want big, bloated libraries like Qt or Gtk or Boost to be dynamically linked, and OS facilities like libc to be statically linked.

> I want big, bloated libraries like Qt or Gtk or Boost to be dynamically linked

These fall under "GUI" in the "base system" category in my opinion.

> and OS facilities like libc to be statically linked.

If you're not being sarcastic right now, I'm really curious as to your reasoning.

Re: Oasis: a small statically-linked Linux system

#79

Post by nmz »

arendtio wrote: post #39
gameswithgo wrote: post #15

Earlier quoted context omitted.

As someone who uses linux casually, I often experience pain when I download a program, try to run it, and it fails because some dependency is not installed or is not the right version. So you try to figure out how to install it and it in turn needs a couple of things. All of this works fine if whatever package manager your distro uses has the program you want, but the package managers don't have evertying! So I'd lov…

> So I'd love an ecosystem where things tend to be statically linked unless there is a good reason not to. Well, every time there is a bug fix in one of your dependencies you/your distribution would have to recompile everything that depends on it. Which will use quite some resources. So to actually use this you would have to reinstall (nearly) all programs once a week or so (no matter if source or binary distribution…

So long as I'm not the one building it, its fine. My internet can redownload everything. It also depends on what libraries are being used. musl does generate smaller binaries than glibc. and if you look at the massive project that is sqlite, building software should only take a couple of seconds. Of course, whether anyone actually goes to such lengths for easy compilation is another matter entirely.

https://sta.li/

Re: Oasis: a small statically-linked Linux system

#80

Post by pjmlp »

Having grown up on a static linked world, where dynamic linking was a thing of big iron computers that we dreamt about being able to do in home computers this trend of static linked compiled stuff feels somehow a tragedy of some sort.

How bad have we gone, that is now trendy to return to the days of static compiled binaries and process IPC to achieve any sort of dynamism.

Post reply on HN