Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The state of binary compatibility on Linux and how to address it

jangafx.com

Post reply on HN
Print view View original

11–20 of 145 posts

Re: The state of binary compatibility on Linux and how to address it

#11

Post by IX-103 »

sylware wrote: post #2

This article missed a critical point which is "the right way" to select a glibc ABI version: see binutils ld documentation, second part of the page related to VERSION support. This must include glibc internal symbols. This will allow to craft ELF binaries on a modern distro which will run on "older" distros. This is critical for games and game engines. There is an significant upfront only-once work in order to select…

Could you point to an example of a project doing it the right way? I'm a little unclear on the type of incantation needed for the "version script".

Re: The state of binary compatibility on Linux and how to address it

#12

Post by AshamedCaptain »

api wrote: post #3
sylware wrote: post #2

This article missed a critical point which is "the right way" to select a glibc ABI version: see binutils ld documentation, second part of the page related to VERSION support. This must include glibc internal symbols. This will allow to craft ELF binaries on a modern distro which will run on "older" distros. This is critical for games and game engines. There is an significant upfront only-once work in order to select…

Is there a reason glibc can't just do a better job at keeping some legacy symbols around? It's not like it's big stuff. They're things like legacy string functions. We're talking a few kilobytes of code in most cases. The Linux kernel goes to a lot of effort to not break user space, at least for non-exotic core features and syscalls. It seems like a lot of user-space in Linux-land does not make the same effort. It's…

The problem is the opposite: they are trying to run executables built using a newer glibc in a system that has an older glibc. glibc keeps all the old function definitions since practically forever.

Frankly, I do not understand who would think glibc symbols themselves would be the challenge in this case. Even if you statically link glibc there's zero guarantee the syscalls will be present in the older Linux (cue .ABI-tag failures). Or even damn ELF format changes (e.g. gnu-style hashes). The simple solution is to build in the older Linux (&glibc).

In my long experience with ancient binaries, glibc has almost never been the problem, and its ability to _run_ ancient binaries is all but excellent; even Linux is more of a problem than glibc is (for starters paths to everywhere in /proc, /sys change every other half-decade).

Re: The state of binary compatibility on Linux and how to address it

#13

Post by rjsw »

vlovich123 wrote: post #8
sylware wrote: post #2

This article missed a critical point which is "the right way" to select a glibc ABI version: see binutils ld documentation, second part of the page related to VERSION support. This must include glibc internal symbols. This will allow to craft ELF binaries on a modern distro which will run on "older" distros. This is critical for games and game engines. There is an significant upfront only-once work in order to select…

Which works if you use binutils ld. Does it work with mold or gold? And then how do you use this with languages other than c++/c like Go or Rust?

I thought that Go invoked syscalls directly instead of going through libc.

Re: The state of binary compatibility on Linux and how to address it

#14

Post by Jeaye »

I don't understand why they don't just statically link their binaries. First, they said this:

> Even if you managed to statically link GLIBC—or used an alternative like musl—your application would be unable to load any dynamic libraries at runtime.

But then they immediately said they actually statically link all of their deps aside from libc.

> Instead, we take a different approach: statically linking everything we can.

If they're statically linking everything other than libc, then using musl or statically linking glibc will finish the job. Unless they have some need for loading share libs at runtime which they didn't already have linked into their binary (i.e. manual dlopen), this solves the portability problem on Linux.

What am I missing (assuming I know of the security implications of statically linked binaries -- which they didn't mention as a concern)?

Re: The state of binary compatibility on Linux and how to address it

#15

Post by bee_rider »

Really the title should be something like “the difficulty of releasing binaries on Linux and how to work around it.” It isn’t really an atrocious state, the goal of a distro should be to get useful software into the repos. Software not distributed in an open source format doesn’t really help there.

Re: The state of binary compatibility on Linux and how to address it

#16

Post by masfuerte »

Jeaye wrote: post #14

I don't understand why they don't just statically link their binaries. First, they said this: > Even if you managed to statically link GLIBC—or used an alternative like musl—your application would be unable to load any dynamic libraries at runtime. But then they immediately said they actually statically link all of their deps aside from libc. > Instead, we take a different approach: statically linking everything we c…

Various things including name (DNS) resolution rely on dynamic linking.

Re: The state of binary compatibility on Linux and how to address it

#17

Post by Jeaye »

masfuerte wrote: post #16
Jeaye wrote: post #14

I don't understand why they don't just statically link their binaries. First, they said this: > Even if you managed to statically link GLIBC—or used an alternative like musl—your application would be unable to load any dynamic libraries at runtime. But then they immediately said they actually statically link all of their deps aside from libc. > Instead, we take a different approach: statically linking everything we c…

Various things including name (DNS) resolution rely on dynamic linking.

Are you saying that a statically linked binary cannot make an HTTP request to `google.com` because it would be unable to resolve the domain name?

There are entire distros, like alpine, built on musl. I find this very hard to believe.

Re: The state of binary compatibility on Linux and how to address it

#18

Post by forrestthewoods »

Jeaye wrote: post #14

I don't understand why they don't just statically link their binaries. First, they said this: > Even if you managed to statically link GLIBC—or used an alternative like musl—your application would be unable to load any dynamic libraries at runtime. But then they immediately said they actually statically link all of their deps aside from libc. > Instead, we take a different approach: statically linking everything we c…

musl and glibc static links are their own Pandora’s box of pain and suffering. They don’t “just work” like you’d hope and dream.

Re: The state of binary compatibility on Linux and how to address it

#19

Post by forrestthewoods »

AndyKelley wrote: post #7
sylware wrote: post #2

This article missed a critical point which is "the right way" to select a glibc ABI version: see binutils ld documentation, second part of the page related to VERSION support. This must include glibc internal symbols. This will allow to craft ELF binaries on a modern distro which will run on "older" distros. This is critical for games and game engines. There is an significant upfront only-once work in order to select…

One of the features Zig provides is ability to target any glibc version. See https://github.com/ziglang/glibc-abi-tool/ for more details on how this is solved.

You’re my hero for this.

One of my side projects is building a toolchain to enable C++ cross-compile using the Zig header/source libs.

I didn’t love Zig as a Clang++ replacement because it has a bit too much magic. And it might go ahead? But the underlying library code is a God send

Re: The state of binary compatibility on Linux and how to address it

#20

Post by cozzyd »

FitCodIa wrote: post #9
api wrote: post #3

Earlier quoted context omitted.

Is there a reason glibc can't just do a better job at keeping some legacy symbols around? It's not like it's big stuff. They're things like legacy string functions. We're talking a few kilobytes of code in most cases. The Linux kernel goes to a lot of effort to not break user space, at least for non-exotic core features and syscalls. It seems like a lot of user-space in Linux-land does not make the same effort. It's…

> a lot of user-space in Linux-land does not make the same effort I believe that, what the article misses is that glibc is maintained and extended with an entirely different community and development model. Windows remains compatible over decades because Microsoft (a) is the sole distributor, and (b) puts an immense effort towards backwards compat. In Linux userspace, it's simply a non-goal across distributions. If y…

The big Linux distros (EL, Ubuntu) in fact have the opposite incentive, to get proprietary vendors to rather their distribution specifically.
Post reply on HN