Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

github.com

Post reply on HN
Print view View original

191–200 of 215 posts

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#191

Post by lmm »

AshamedCaptain wrote: post #154
drowsspa wrote: post #116

Earlier quoted context omitted.

I find it amazing how much the mess that building C/C++ code has been for so many decades seems to have influenced the direction technology, the economy and even politics has been going. Really, what would the world look like if this problem had been properly solved? Would the centralization and monetization of the Internet have followed the same path? Would Windows be so dominant? Would social media have evolved to…

What I find amazing is why people continously claim glibc is the problem here. I have a commercial software binary from 1996 that _still works_ to this day. It even links with X11, and works under Xwayland. The trick? It's not statically linked, but dynamically linked. And it doesn't like with anything other than glibc, X11 ... and bdb. At this point I think people just do not know how binary compatibility works at a…

> The trick? It's not statically linked, but dynamically linked. And it doesn't like with anything other than glibc, X11 ... and bdb.

How would that work given that glibc has gone through a soname change since then? If it's from 1996 are you sure the secret isn't that it uses non-g libc?

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#193

Post by renehsz »

AshamedCaptain wrote: post #162
marcosdumay wrote: post #161

Earlier quoted context omitted.

The problem of modern libc (newer than ~2004, I have no idea what that 1996 one is doing) isn't that old software stops working. It's that you can't compile software on your up to date desktop and have it run on your "security updates only" server. Or your clients "couple of years out of date" computers. And that doesn't require using newer functionality.

But this is not "backwards compatibility". No one promises this type of "forward compatibility" that you are asking for . Even win32 only does it exceptionally... maybe today you can still build a win10 binary with a win11 toolchain, but you cannot build a win98 binary with it for sure. And this has nothing to do with 1996, or 2004 glibc at all. In fact, glibc makes this otherwise impossible task actually possible: y…

> maybe today you can still build a win10 binary with a win11 toolchain, but you cannot build a win98 binary with it for sure.

In my experience, that's not quite accurate. I'm working on a GUI program that targets Windows NT 4.0, built using a Win11 toolchain. With a few tweaks here and there, it works flawlessly. Microsoft goes to great lengths to keep system DLLs and the CRT forward- and backward-compatible. It's even possible to get libc++ working: https://building.enlyze.com/posts/targeting-25-years-of-wind...

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#194

Post by VadimPR »

account42 wrote: post #50
VadimPR wrote: post #31

Earlier quoted context omitted.

We fix this issue by distributing ours in a tar file with the executable bit set. Linux novices can just double click on the tar to exact it and double click again on the actual appimage. Been doing it this way for years now, so it's well battle tested.

That kind of defeats the point of an AppImage though - you could just as well have a tar archive with a c classic collection of binaries + optional launcher script.

A single file is much better to manage on the eyes than a whole bunch of them, plus AppImages can be installed into the desktop using integration.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#195

Post by flohofwoe »

sebastos wrote: post #173
flohofwoe wrote: post #62

Earlier quoted context omitted.

Dynamic libraries make a lot of sense as operating system interface when they guarantee a stable API and ABI (see Windows for how to do that) - the other scenarios where DLLs make sense is for plugin systems. But that's pretty much it, for anything else static linking is superior because it doesn't present an optimization barrier (especially for dead code elimination). No idea why the glibc can't provide API+ABI stab…

Genuine question - are there examples (research? old systems?) of the interface to the operating system being exposed differently than a library? How might that work exactly?

> examples ... of the interface to the operating system being exposed differently than a library

Linux syscalls, MS-DOS 'software interrupts'...

But that's not the issue, operating system interfaces can be exposed via DLLs, those DLLs interfaces just must be guaranteed to be stable (like on Windows).

Tbh, I'm not sure why I can't simply tell the gcc linker some random old glibc version number from the late 1990s and the gcc linker checks whether I'm using any functions that haven't been available in that old version (and in that case errors out). That would be the most frictionless solution, and surely it can't be too hard to annotate glibc functions with a version number in the gcc system headers when that function first appeared.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#196

Post by throwaway2046 »

diego_sandoval wrote: post #190
vv_ wrote: post #38

Earlier quoted context omitted.

It's easier to distribute software fully self-contained, if you ignore the pain of statically linking everything together :)

What's the pain?

I'm guessing the pain of fighting the various build systems that insist on dynamic linking, sometimes against the user's explicit wishes.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#197

Post by saagarjha »

AshamedCaptain wrote: post #162
marcosdumay wrote: post #161

Earlier quoted context omitted.

The problem of modern libc (newer than ~2004, I have no idea what that 1996 one is doing) isn't that old software stops working. It's that you can't compile software on your up to date desktop and have it run on your "security updates only" server. Or your clients "couple of years out of date" computers. And that doesn't require using newer functionality.

But this is not "backwards compatibility". No one promises this type of "forward compatibility" that you are asking for . Even win32 only does it exceptionally... maybe today you can still build a win10 binary with a win11 toolchain, but you cannot build a win98 binary with it for sure. And this has nothing to do with 1996, or 2004 glibc at all. In fact, glibc makes this otherwise impossible task actually possible: y…

This kind of compatibility is available on, of all systems, macOS.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#198

Post by drowsspa »

joshmarinacci wrote: post #123
drowsspa wrote: post #116

Earlier quoted context omitted.

I find it amazing how much the mess that building C/C++ code has been for so many decades seems to have influenced the direction technology, the economy and even politics has been going. Really, what would the world look like if this problem had been properly solved? Would the centralization and monetization of the Internet have followed the same path? Would Windows be so dominant? Would social media have evolved to…

How does this technical issue affect the economy and politics? In what way would the world be different just because we used a better linker?

That was the point of the successive questions in the second paragraph...

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#199

Post by MrDrMcCoy »

fdgdd wrote: post #177
amelius wrote: post #5

Is there a tool that takes an executable, collects all the required .so files and produces either a static executable, or a package that runs everywhere?

Exodus ( https://github.com/intoli/exodus ) used to be good for this but is giving me a Python error these days.

A couple similar projects:

https://github.com/sigurd-dev/mkblob https://github.com/tweag/clodl

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#200

Post by AshamedCaptain »

renehsz wrote: post #193
AshamedCaptain wrote: post #162

Earlier quoted context omitted.

But this is not "backwards compatibility". No one promises this type of "forward compatibility" that you are asking for . Even win32 only does it exceptionally... maybe today you can still build a win10 binary with a win11 toolchain, but you cannot build a win98 binary with it for sure. And this has nothing to do with 1996, or 2004 glibc at all. In fact, glibc makes this otherwise impossible task actually possible: y…

> maybe today you can still build a win10 binary with a win11 toolchain, but you cannot build a win98 binary with it for sure. In my experience, that's not quite accurate. I'm working on a GUI program that targets Windows NT 4.0, built using a Win11 toolchain. With a few tweaks here and there, it works flawlessly. Microsoft goes to great lengths to keep system DLLs and the CRT forward- and backward-compatible. It's e…

What does "a Win11 toolchain" mean here? In the article you link, the guy is filling missing functions, rewriting the runtime, and overall doing even more work than what I need to do to build binaries on a Linux system from 2026 that would work on a Linux from the 90s : a simple chroot. Even building gcc is a walk in the park compared to reimplementing OS threading functions...
Post reply on HN