Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The state of binary compatibility on Linux and how to address it

jangafx.com

Post reply on HN
Print view View original

21–30 of 145 posts

Re: The state of binary compatibility on Linux and how to address it

#21

Post by forrestthewoods »

AshamedCaptain wrote: post #12
api wrote: post #3

Earlier quoted context omitted.

Is there a reason glibc can't just do a better job at keeping some legacy symbols around? It's not like it's big stuff. They're things like legacy string functions. We're talking a few kilobytes of code in most cases. The Linux kernel goes to a lot of effort to not break user space, at least for non-exotic core features and syscalls. It seems like a lot of user-space in Linux-land does not make the same effort. It's…

The problem is the opposite: they are trying to run executables built using a newer glibc in a system that has an older glibc. glibc keeps all the old function definitions since practically forever. Frankly, I do not understand who would think glibc symbols themselves would be the challenge in this case. Even if you statically link glibc there's zero guarantee the syscalls will be present in the older Linux (cue .ABI…

> executables built and using a newer glibc

It’s an abomination that Linux uses system libraries when building. Catastrophically terrible and stupid decision.

It should be trivial for any program to compile and specify any arbitrary previous version of glibc as the target.

Linux got this so incredibly wildly wrong. It’s a shame.

Re: The state of binary compatibility on Linux and how to address it

#22

Post by masfuerte »

Jeaye wrote: post #17
masfuerte wrote: post #16

Earlier quoted context omitted.

Various things including name (DNS) resolution rely on dynamic linking.

Are you saying that a statically linked binary cannot make an HTTP request to `google.com` because it would be unable to resolve the domain name? There are entire distros, like alpine, built on musl. I find this very hard to believe.

The configuration of DNS resolution on Linux is quite complicated [1]. Musl just ignores all that. You can build a distro that works with musl, but a static musl binary dropped into an arbitrary Linux system won't necessarily work correctly.

[1]: https://news.ycombinator.com/item?id=43451861

Re: The state of binary compatibility on Linux and how to address it

#23

Post by en4bz »

Jeaye wrote: post #14

I don't understand why they don't just statically link their binaries. First, they said this: > Even if you managed to statically link GLIBC—or used an alternative like musl—your application would be unable to load any dynamic libraries at runtime. But then they immediately said they actually statically link all of their deps aside from libc. > Instead, we take a different approach: statically linking everything we c…

OpenGL and Vulkan are provided by the GPU vendor you can't statically link them.

Re: The state of binary compatibility on Linux and how to address it

#24

Post by Spivak »

Jeaye wrote: post #17
masfuerte wrote: post #16

Earlier quoted context omitted.

Various things including name (DNS) resolution rely on dynamic linking.

Are you saying that a statically linked binary cannot make an HTTP request to `google.com` because it would be unable to resolve the domain name? There are entire distros, like alpine, built on musl. I find this very hard to believe.

You have to bundle your own resolver into your application. But here's the rub, users expect your application to respect nsswitch which requires loading shared libs which execute arbitrary code. How Go handles this is somewhat awkward. They parse /etc/nsswitch and decide if they can cheat and use their own resolver based on what modules they see[1]. Otherwise they farm out to cgo to go through glibc.

[1] They're playing with fire here because you can't really assume to know for sure how the module 'dns' behaves. A user could replace the lib that backs it with their own that resolves everything to zombo.com. It would be one thing if nsswitch described behavior which was well defined and could be emulated but it doesn't, it specifies a specific implementation.

Re: The state of binary compatibility on Linux and how to address it

#25

Post by AshamedCaptain »

Jeaye wrote: post #14

I don't understand why they don't just statically link their binaries. First, they said this: > Even if you managed to statically link GLIBC—or used an alternative like musl—your application would be unable to load any dynamic libraries at runtime. But then they immediately said they actually statically link all of their deps aside from libc. > Instead, we take a different approach: statically linking everything we c…

And please, statically linking everything is NOT a solution -- the only reason I can run some games from 20 years ago still on my recent Linux is because they didn't decide to stupidly statically link everything, so I at least _can_ replace the libraries with hooks that make the games work with newer versions.

Re: The state of binary compatibility on Linux and how to address it

#26

Post by Jeaye »

Spivak wrote: post #24
Jeaye wrote: post #17

Earlier quoted context omitted.

Are you saying that a statically linked binary cannot make an HTTP request to `google.com` because it would be unable to resolve the domain name? There are entire distros, like alpine, built on musl. I find this very hard to believe.

You have to bundle your own resolver into your application. But here's the rub, users expect your application to respect nsswitch which requires loading shared libs which execute arbitrary code. How Go handles this is somewhat awkward. They parse /etc/nsswitch and decide if they can cheat and use their own resolver based on what modules they see[1]. Otherwise they farm out to cgo to go through glibc. [1] They're play…

Fascinating. Thanks for breaking this down more. I think the article could've explained this point further.

Re: The state of binary compatibility on Linux and how to address it

#27

Post by jfuafdfwa »

rjsw wrote: post #13
vlovich123 wrote: post #8

Earlier quoted context omitted.

Which works if you use binutils ld. Does it work with mold or gold? And then how do you use this with languages other than c++/c like Go or Rust?

I thought that Go invoked syscalls directly instead of going through libc.

It's Go's best feature, a simple CGO_ENABLED=0 gives you freedom from the tyranny of libc.

Re: The state of binary compatibility on Linux and how to address it

#28

Post by declan_roberts »

AshamedCaptain wrote: post #25
Jeaye wrote: post #14

I don't understand why they don't just statically link their binaries. First, they said this: > Even if you managed to statically link GLIBC—or used an alternative like musl—your application would be unable to load any dynamic libraries at runtime. But then they immediately said they actually statically link all of their deps aside from libc. > Instead, we take a different approach: statically linking everything we c…

And please, statically linking everything is NOT a solution -- the only reason I can run some games from 20 years ago still on my recent Linux is because they didn't decide to stupidly statically link everything, so I at least _can_ replace the libraries with hooks that make the games work with newer versions.

How do you troubleshoot and figure that out?

Re: The state of binary compatibility on Linux and how to address it

#29

Post by o11c »

Jeaye wrote: post #17
masfuerte wrote: post #16

Earlier quoted context omitted.

Various things including name (DNS) resolution rely on dynamic linking.

Are you saying that a statically linked binary cannot make an HTTP request to `google.com` because it would be unable to resolve the domain name? There are entire distros, like alpine, built on musl. I find this very hard to believe.

All versions of MUSL prior to 1.2.4 (released less than two years ago) would indeed fail to perform DNS lookups in many common cases, and a lot of programs could not run in MUSL as a result. (I'm not aware of what specific deficiencies remain in MUSL, but given the history even when there are explicit standards, I am confident that there are more.) This wasn't related to dynamic linking though.

Glibc's NSS is mostly relevant for LANs. Which is a lot of corporate and home networks.

Re: The state of binary compatibility on Linux and how to address it

#30

Post by AshamedCaptain »

forrestthewoods wrote: post #21
AshamedCaptain wrote: post #12

Earlier quoted context omitted.

The problem is the opposite: they are trying to run executables built using a newer glibc in a system that has an older glibc. glibc keeps all the old function definitions since practically forever. Frankly, I do not understand who would think glibc symbols themselves would be the challenge in this case. Even if you statically link glibc there's zero guarantee the syscalls will be present in the older Linux (cue .ABI…

> executables built and using a newer glibc It’s an abomination that Linux uses system libraries when building. Catastrophically terrible and stupid decision. It should be trivial for any program to compile and specify any arbitrary previous version of glibc as the target. Linux got this so incredibly wildly wrong. It’s a shame.

Ehm... you _can_ use the older toolchain even in the newer "Linux", in the same way you have to use a older Visual Studio to target Windows 8 from Windows 10.
Post reply on HN