Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

github.com

Post reply on HN
Print view View original

121–130 of 215 posts

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#121

Post by mdavid626 »

toast0 wrote: post #119
mdavid626 wrote: post #55

Earlier quoted context omitted.

Yes, that's true. But I'm always a bit sceptical about such approaches. They are not universal. You still need glibc/musl to be the same on the target system. Also, if you compile againt new glibc version, but try to run on old glibc version, it might not work. These are just strange and confusing from the end users' perspective.

> But I'm always a bit sceptical about such approaches. They are not universal. You still need glibc/musl to be the same on the target system. Also, if you compile againt new glibc version, but try to run on old glibc version, it might not work. Why would you include most of your dynamic libraries but not your libc? You could still run into problems if you (or your libraries) want to use syscalls that weren't availab…

You can include it, but

- either you use chroot, proot or similar to make /lib path contain your executable’s loader

- or you hardcode different loader path into your executable

Both are difficult for an end user.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#122

Post by holowoodman »

direwolf20 wrote: post #113
holowoodman wrote: post #85

Earlier quoted context omitted.

AMD uses the dynamically linked system libGL.so, usually Mesa.

So you still need dynamic linking to load the right driver for your graphics card.

Most stuff like that uses some kind of "icd" mechanism that does 'dlopen' on the vendor-specific parts of the library. Afaik neither OpenGL nor Vulkan nor OpenCL are usable without at least dlopen, if not full dynamic linking.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#123

Post by joshmarinacci »

drowsspa wrote: post #116
Rochus wrote: post #60

So what we need is essentially a "libc virtualization". But Musl is only available on Linux, isn't it? Cosmopolitan ( https://github.com/jart/cosmopolitan ) goes further and is available also on Mac and Windows, and it uses e.g. SIMD and other performance related improvements. Unfortunately, one has to cut through the marketing "magic" to find the main engineering value; stripping away the "polyglot" shell-script hac…

I find it amazing how much the mess that building C/C++ code has been for so many decades seems to have influenced the direction technology, the economy and even politics has been going. Really, what would the world look like if this problem had been properly solved? Would the centralization and monetization of the Internet have followed the same path? Would Windows be so dominant? Would social media have evolved to…

How does this technical issue affect the economy and politics? In what way would the world be different just because we used a better linker?

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#124

Post by holowoodman »

electroly wrote: post #99
rlpb wrote: post #95

Earlier quoted context omitted.

Imagine a fully statically linked version of Debian. What happens when there’s a security update in a commonly used library? Am I supposed to redownload a rebuild of basically the entire distro every time this happens, or else what?

Steel-manning the idea, perhaps they would ship object files (.o/.a) and the apt-get equivalent would link the system? I believe this arrangement was common in the days before dynamic linking. You don't have to redownload everything, but you do have to relink everything.

But if I have to relink everything, I need all the makefiles, linker scripts and source code structure. I might as well compile it outright. On the other hand, I might as well just link it whenever I run it, like, dynamically ;)

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#125

Post by aspbee555 »

I managed to get this combo going not too long ago with my musl rust app but I found that even after it all was compiled and loading the lib it did not function properly because the library I loaded still depended on libc functions. even with everything compiled into a huge monolithic musl binary it couldn't find something graphics related

I eventually decided to keep the tiny musl app and make a companion app in a secondary process as needed (since the entire point of me compiling musl was cross platform linux compatibility/stability)

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#126

Post by toast0 »

mdavid626 wrote: post #121
toast0 wrote: post #119

Earlier quoted context omitted.

> But I'm always a bit sceptical about such approaches. They are not universal. You still need glibc/musl to be the same on the target system. Also, if you compile againt new glibc version, but try to run on old glibc version, it might not work. Why would you include most of your dynamic libraries but not your libc? You could still run into problems if you (or your libraries) want to use syscalls that weren't availab…

You can include it, but - either you use chroot, proot or similar to make /lib path contain your executable’s loader - or you hardcode different loader path into your executable Both are difficult for an end user.

This isn't that hard (that's not to say this is easy, it is tricky). Your executable should be a statically linked stub loader with an awful lot of data, the stub loader dynamically links your real executable (and libraries, including libc) from the data and runs it.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#127

Post by [deleted] »

leni536 wrote: post #104

Do I get this right that this effectively dlopens glibc (indirectly) into an executable that is statically linked to musl? How can the two runtimes coexist? What about malloc/free? AFAIK both libc's allocators take ownership of brk, that can't be good. What about malloc/free across the dynamic library interface? There are certainly libraries that hand out allocated objects and expect the user to free them, but that's…

[deleted]

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#129

Post by yjftsjthsd-h »

oguz-ismail2 wrote: post #111
yjftsjthsd-h wrote: post #110

Earlier quoted context omitted.

>> Linux > if you configure binfmt_misc I don't think that's a requirement, it'll just fall back to the shell script bootstrap without it.

On some distros, yes. On others it'll fire up Wine for whatever reason

Okay, yes, if you configure binfmt_misc for WINE and not APE then PE-compatible binaries will get run with WINE and not APE. That feels unfair.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#130

Post by paddim8 »

rlpb wrote: post #95
arghwhat wrote: post #47

Earlier quoted context omitted.

Dynamic libraries have been frowned upon since their inception as being a terrible solution to a non-existent problem, generally amplifying binary sizes and harming performance. Some fun quotes of quite notable characters on the matter here: https://harmful.cat-v.org/software/dynamic-linking/ In practice, a statically linked system is often smaller than a meticulously dynamically linked one - while there are many cop…

Imagine a fully statically linked version of Debian. What happens when there’s a security update in a commonly used library? Am I supposed to redownload a rebuild of basically the entire distro every time this happens, or else what?

Then you update those dependencies. Not very difficult with a package manager. And most dependencies aren't used by a ton of programs in a single system anyway. It is not a big deal in practice.
Post reply on HN