Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The state of binary compatibility on Linux and how to address it

jangafx.com

Post reply on HN
Print view View original

31–40 of 145 posts

Re: The state of binary compatibility on Linux and how to address it

#31

Post by klysm »

AshamedCaptain wrote: post #25
Jeaye wrote: post #14

I don't understand why they don't just statically link their binaries. First, they said this: > Even if you managed to statically link GLIBC—or used an alternative like musl—your application would be unable to load any dynamic libraries at runtime. But then they immediately said they actually statically link all of their deps aside from libc. > Instead, we take a different approach: statically linking everything we c…

And please, statically linking everything is NOT a solution -- the only reason I can run some games from 20 years ago still on my recent Linux is because they didn't decide to stupidly statically link everything, so I at least _can_ replace the libraries with hooks that make the games work with newer versions.

Software running for 20 years is not always a reasonable requirement.

Re: The state of binary compatibility on Linux and how to address it

#32

Post by James_K »

Here's a thought: just distribute source code. ABI issues should be mostly fixed. Most computers can compile source code fast enough for the user not to notice and cache the results so that it's never a problem again. If you want optimised code, you can do a source to source optimisation then zip and minify the file. You could compile such a library to approximately native speeds without much user-end lag using modern JIT methods, and maybe even run LTO in a background thread so that the exectuables outdo dynamically linked ones.

Re: The state of binary compatibility on Linux and how to address it

#33

Post by lukeh »

Windows having multiple C libraries has its own pain points, in particular it's difficult to ship binary libraries that return allocated memory to their consumer (you either need to have the library consumer allocate the memory, which probably explains why so many Win32 APIs have this behaviour, or allow alloc/free functions to be registered). Not to mention different C libraries having their own file handle, TLS, etc state. Unsurprisingly Microsoft now ships the Universal CRT (UCRT) as part of Windows.

Re: The state of binary compatibility on Linux and how to address it

#34

Post by evidencetamper »

AshamedCaptain wrote: post #25
Jeaye wrote: post #14

I don't understand why they don't just statically link their binaries. First, they said this: > Even if you managed to statically link GLIBC—or used an alternative like musl—your application would be unable to load any dynamic libraries at runtime. But then they immediately said they actually statically link all of their deps aside from libc. > Instead, we take a different approach: statically linking everything we c…

And please, statically linking everything is NOT a solution -- the only reason I can run some games from 20 years ago still on my recent Linux is because they didn't decide to stupidly statically link everything, so I at least _can_ replace the libraries with hooks that make the games work with newer versions.

As long as the library is available.

Neither static nor dynamic linking is looking to solve the 20 year old binaries issue, so both will have different issues.

But I think it's easier for me to find a 20 year old ISO of a Red Hat/Slackware where I can simply run the statically linked binary. Dependency hell for older distros become really difficult when the older packages are not archived anywhere anymore.

Re: The state of binary compatibility on Linux and how to address it

#35

Post by Lerc »

Jeaye wrote: post #14

I don't understand why they don't just statically link their binaries. First, they said this: > Even if you managed to statically link GLIBC—or used an alternative like musl—your application would be unable to load any dynamic libraries at runtime. But then they immediately said they actually statically link all of their deps aside from libc. > Instead, we take a different approach: statically linking everything we c…

Dynamic linking obviously has benefits or there would not be any incentive to build them or provide the capacity for them.

The problem is they also have problems which motivates people to statically link.

I remember back in the Amiga days when there were multiple libraries that provided file requesters. At one point I saw a unifying file requester library that implemented the interfaces of multiple others so that all requesters had the same look.

It's something that hasn't been done as far as I am aware on Linux. partially because of the problems with Linux dynamic libraries.

I think the answer isn't just static linking.

I think the solution is a commitment.

If you are going to make a dynamic library, commit to backwards compatibility. If you can't provide that, that's ok, but please statically link.

Perhaps making a library at a base level with a forever backwards compatible interface with a static version for breaking changes would help. That might allow for a blend of bug support and adding future features.

Re: The state of binary compatibility on Linux and how to address it

#36

Post by BwackNinja »

There is no distinction between system and program libraries in Linux. We used to pretend there was one before usrmigration, but that was never good to take seriously.

The distro as packager model ensures that everything is mixed together in the filesystem and is actively hostile to external packaging. Vendoring dependencies or static linking improves compatibility by choosing known working versions, but decreases incentive and ability for downstream (or users) to upgrade those dependencies.

The libc stuff in this article is mostly glibc-specific, and you'd have fewer issues targeting musl. Mixing static linking and dlopen doesn't make much sense, as said here[1] which is an interesting thread. Even dns resolution on glibc implies dynamic linking due to nsswitch.

Solutions like Snap, Flatpak, and AppImage work to contain the problem by reusing the same abstractions internally rather than introducing anything that directly addresses the issue. We won't have a clean solution until we collectively abandon the FHS for a decentralized filesystem layout where adding an application (not just a program binary) is as easy as extracting a package into a folder and integrates with the rest of the system. I've worked on this off and on for a while, but being so opinionated makes everything an uphill battle while accepting the current reality is easy.

[1] https://musl.openwall.narkive.com/lW4KCyXd/static-linking-an...

Re: The state of binary compatibility on Linux and how to address it

#37

Post by nobodyandproud »

Lerc wrote: post #35
Jeaye wrote: post #14

I don't understand why they don't just statically link their binaries. First, they said this: > Even if you managed to statically link GLIBC—or used an alternative like musl—your application would be unable to load any dynamic libraries at runtime. But then they immediately said they actually statically link all of their deps aside from libc. > Instead, we take a different approach: statically linking everything we c…

Dynamic linking obviously has benefits or there would not be any incentive to build them or provide the capacity for them. The problem is they also have problems which motivates people to statically link. I remember back in the Amiga days when there were multiple libraries that provided file requesters. At one point I saw a unifying file requester library that implemented the interfaces of multiple others so that all…

At least for some apps, perhaps it’s Wine and the Win32 API which is the answer.

Re: The state of binary compatibility on Linux and how to address it

#38

Post by zer0zzz »

Jeaye wrote: post #14

I don't understand why they don't just statically link their binaries. First, they said this: > Even if you managed to statically link GLIBC—or used an alternative like musl—your application would be unable to load any dynamic libraries at runtime. But then they immediately said they actually statically link all of their deps aside from libc. > Instead, we take a different approach: statically linking everything we c…

Static linking makes it impossible for a library to evolve external to applications. That’s not a great outcome for a lot of reasons.

Re: The state of binary compatibility on Linux and how to address it

#39

Post by jll29 » 

  > More importantly, separating the dynamic linker from the C library itself would allow multiple versions of libc to coexist, eliminating a major source of compatibility issues. This is exactly how Windows handles it, which is one of the reasons Windows maintains such strong binary compatibility. You can still run decades-old Windows software today because Microsoft doesn’t force everything to be tied to a single, ever-changing libc.
One of the questions of multiple versions on the same box is what about security issues of those older versions...

Re: The state of binary compatibility on Linux and how to address it

#40

Post by moron4hire »

jll29 wrote: post #39

> More importantly, separating the dynamic linker from the C library itself would allow multiple versions of libc to coexist, eliminating a major source of compatibility issues. This is exactly how Windows handles it, which is one of the reasons Windows maintains such strong binary compatibility. You can still run decades-old Windows software today because Microsoft doesn’t force everything to be tied to a single, ev…

What about the security issues of the old operating systems people keep around for their mission critical software that has no upgrade path?
Post reply on HN