Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

github.com

Post reply on HN
Print view View original

11–20 of 215 posts

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#11

Post by TheDong »

amelius wrote: post #5

Is there a tool that takes an executable, collects all the required .so files and produces either a static executable, or a package that runs everywhere?

There are things like this.

The things I know of and can think of off the top of my head are:

1. appimage https://appimage.org/

2. nix-bundle https://github.com/nix-community/nix-bundle

3. guix via guix pack

4. A small collection of random small projects hardly anyone uses for docker to do this (i.e. https://github.com/NilsIrl/dockerc )

5. A docker image (a package that runs everywhere, assuming a docker runtime is available)

6. https://flatpak.org/

7. https://en.wikipedia.org/wiki/Snap_(software)

AppImage is the closest to what you want I think.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#12

Post by sambuccid »

einpoklum wrote: post #2

This seems interesting even regardless of go. Is it realistic to create an executable which would work on very different kinds of Linux distros? e.g. 32-bit and 64-bit? Or maybe some general framework/library for building an arbitrary program at least for "any libc"?

Appimage exists that packs linux applications into a single executable file that you just download and open. It works on most linux distros

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#13

Post by greyw »

sambuccid wrote: post #12
einpoklum wrote: post #2

This seems interesting even regardless of go. Is it realistic to create an executable which would work on very different kinds of Linux distros? e.g. 32-bit and 64-bit? Or maybe some general framework/library for building an arbitrary program at least for "any libc"?

Appimage exists that packs linux applications into a single executable file that you just download and open. It works on most linux distros

I vaguely remember that Appimage-based programs would fail for me because of fuse and glibc symbol version incompatibilties.

Gave up them afterwards. If I need to tweak dependencies might as well deal with the packet manager of my distro.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#14

Post by aa-jv »

amelius wrote: post #5

Is there a tool that takes an executable, collects all the required .so files and produces either a static executable, or a package that runs everywhere?

AppImage comes close to fulfilling this need:

https://appimage.github.io/appimagetool/

Myself, I've committed to using Lua for all my cross-platform development needs, and in that regard I find luastatic very, very useful ..

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#15

Post by iberator »

einpoklum wrote: post #2

This seems interesting even regardless of go. Is it realistic to create an executable which would work on very different kinds of Linux distros? e.g. 32-bit and 64-bit? Or maybe some general framework/library for building an arbitrary program at least for "any libc"?

Yup. Just compile it as static executable. Static binaries are very undervalued imo.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#16

Post by formerly_proven »

amelius wrote: post #5

Is there a tool that takes an executable, collects all the required .so files and produces either a static executable, or a package that runs everywhere?

I don't think you can link shared objects into a static binary because you'd have to patch all instances where the code reads the PLT/GOT, but this can be arbitrarily mangled by the optimizer, and turn them back into relocations for the linker to then resolve them.

You can change the rpath though, which is sort of like an LD_LIBRARY_PATH baked into the object, which makes it relatively easy to bundle everything but libc with your binary.

edit: Mild correction, there is this: https://sourceforge.net/projects/statifier/ But the way this works is that it has the dynamic linker load everything (without ASLR / in a compact layout, presumably) and then dumps an image of the process. Everything else is just increasingly fancy ways of copying shared objects around and making ld.so prefer the bundled libraries.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#17

Post by flohofwoe »

iberator wrote: post #15
einpoklum wrote: post #2

This seems interesting even regardless of go. Is it realistic to create an executable which would work on very different kinds of Linux distros? e.g. 32-bit and 64-bit? Or maybe some general framework/library for building an arbitrary program at least for "any libc"?

Yup. Just compile it as static executable. Static binaries are very undervalued imo.

The "just" is doing a lot of heavylifting here (as detailed in the article), especially for anything that's not a trivial cmdline tool.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#18

Post by seba_dos1 »

Meneth wrote: post #9

That seems mostly useful for proprietary programs. I don't like it.

Yeah, in my 20 years of using and developing on GNU/Linux the only binary compatibility issues I experienced that I can think of now were related to either Adobe Flash, Adobe Reader or games.

Adobe stuff is of the kind that you'd prefer to not exist at all rather than have it fixed (and today you largely can pretend that it never existed already), and the situation for games has been pretty much fixed by Steam runtimes.

It's fine that some people care about it and some solutions are really clever, but it just doesn't seem to be an actual issue you stumble on in practice much.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#19

Post by dontdoxxme »

quesomaster9000 wrote: post #4
einpoklum wrote: post #2

This seems interesting even regardless of go. Is it realistic to create an executable which would work on very different kinds of Linux distros? e.g. 32-bit and 64-bit? Or maybe some general framework/library for building an arbitrary program at least for "any libc"?

Cosmopolitan goes one further: [binaries] that runs natively on Linux + Mac + Windows + FreeBSD + OpenBSD + NetBSD + BIOS on AMD64 and ARM64 https://justine.lol/cosmopolitan/

Clearly a joke if it uses the .lol tld.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#20

Post by amelius »

TheDong wrote: post #11
amelius wrote: post #5

Is there a tool that takes an executable, collects all the required .so files and produces either a static executable, or a package that runs everywhere?

There are things like this. The things I know of and can think of off the top of my head are: 1. appimage https://appimage.org/ 2. nix-bundle https://github.com/nix-community/nix-bundle 3. guix via guix pack 4. A small collection of random small projects hardly anyone uses for docker to do this (i.e. https://github.com/NilsIrl/dockerc ) 5. A docker image (a package that runs everywhere, assuming a docker runtime is a…

AppImage looks like what I need, thanks.

I wonder though, if I package say a .so file from nVidia, is that allowed by the license?

Post reply on HN