Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The state of binary compatibility on Linux and how to address it

jangafx.com

Post reply on HN
Print view View original

81–90 of 145 posts

Re: The state of binary compatibility on Linux and how to address it

#81

Post by adrian_b »

HKH2 wrote: post #59
ses1984 wrote: post #46

Earlier quoted context omitted.

No, end users need not get involved, it could/should be handled by the operating system.

Gentoo? Compiling big packages takes ages.

More than 20 years, with a single-core Pentium 4, it could take indeed something like 3 days of continuous compilation to compile an entire Gentoo distribution, in order to have a personal computer with every application that one might want.

However, already after the appearance of the first dual-core AMD Athlon64, 20 years ago, that time could be reduced to not much more than a half of day, while nowadays, with a decent desktop CPU from 5 years ago, most Gentoo packages can be compiled and installed in less than a minute.

There are only a few packages whose compilation and installation can take a noticeable time, of up to tens of minutes, depending on the chosen options and on the number of cores of the CPU, e.g. Firefox, LibreOffice, LLVM.

There is only a single package whose compilation may take ages unless you have an expensive CPU and enough memory per core: Google Chromium (including its derivatives that use the same code base).

Re: The state of binary compatibility on Linux and how to address it

#82

Post by adrian_b »

m463 wrote: post #51
James_K wrote: post #32

Here's a thought: just distribute source code. ABI issues should be mostly fixed. Most computers can compile source code fast enough for the user not to notice and cache the results so that it's never a problem again. If you want optimised code, you can do a source to source optimisation then zip and minify the file. You could compile such a library to approximately native speeds without much user-end lag using moder…

the first launch of firefox would take a few hours. let alone the first boot of the linux kernel... :)

This is somewhat exaggerated.

The compilation of firefox could take a few hours on some laptop dual-core Skylake CPU from 10 years ago.

Nowadays, on any decent dektop CPU with many cores the compilation of Firefox should take significantly less than an hour, though it remains one of the handful of open-source applications with a really long and non-negligible compilation time.

The Linux kernel is normally compiled much faster than Firefox, except when one would enable the compilation of all existing kernel modules, for all the hardware that could be supported by Linux, even if almost all of that is not present and it would never be present on the target computer system.

Re: The state of binary compatibility on Linux and how to address it

#83

Post by thyristan »

Jeaye wrote: post #17
masfuerte wrote: post #16

Earlier quoted context omitted.

Various things including name (DNS) resolution rely on dynamic linking.

Are you saying that a statically linked binary cannot make an HTTP request to `google.com` because it would be unable to resolve the domain name? There are entire distros, like alpine, built on musl. I find this very hard to believe.

The easy and conforming way to do that would be to call "getent hosts google.com" and use the answer. But this only works for simple use cases where you just need some IPv4/IPv6 address, you can't get other kinds of DNS records like MX or TLSA this way.

Re: The state of binary compatibility on Linux and how to address it

#84

Post by HexDecOctBin »

mikepavone wrote: post #77
charcircuit wrote: post #72

Earlier quoted context omitted.

>then complain entirely about glibc You can ship all of your of the libraries you use with your executable. This isn't possible to do with glibc. It's the exception which is why it's talked about the most.

> It's the exception which is why it's talked about the most. It's definitely not the only exception. libgl is another obvious example since different GPUs need different userland code. I would be surprised if there there had never been compat regressions in those.

libgl can be dlopened, glibc can't be. That is the problem. If libgl has some incompatibility, I can try to work around that. If glibc has some incompatibility, my executable won't even launch.

I agree with this article completely.

Re: The state of binary compatibility on Linux and how to address it

#85

Post by terinjokes »

pabs3 wrote: post #70
evidencetamper wrote: post #34

Earlier quoted context omitted.

As long as the library is available. Neither static nor dynamic linking is looking to solve the 20 year old binaries issue, so both will have different issues. But I think it's easier for me to find a 20 year old ISO of a Red Hat/Slackware where I can simply run the statically linked binary. Dependency hell for older distros become really difficult when the older packages are not archived anywhere anymore.

Debian archives all of our binaries (and source) here: https://snapshot.debian.org/ Some things built on top of that: https://manpages.debian.org/man/debsnap https://manpages.debian.org/man/debbisect https://wiki.debian.org/BisectDebian https://metasnap.debian.net/ https://reproduce.debian.net/

All since ~mid 2005. Unfortunately a tarball I've been looking for was added before then, but replaced before a release was cut (so it's also missing from those archives).

Re: The state of binary compatibility on Linux and how to address it

#87

Post by seba_dos1 »

HKH2 wrote: post #59
ses1984 wrote: post #46

Earlier quoted context omitted.

No, end users need not get involved, it could/should be handled by the operating system.

Gentoo? Compiling big packages takes ages.

I can effortlessly compile Debian packages on my phone.

With some limits of course. I can't compile Chromium even on my laptop. But most of stuff - I can.

Re: The state of binary compatibility on Linux and how to address it

#88

Post by viraptor »

ianburrell wrote: post #64
gosub100 wrote: post #45

Earlier quoted context omitted.

This blows my mind, that in 2025 we still struggle with a simple task such as "read in a string, parse it, and query a cascade of resolvers to discover it's IP". I just can't fathom how that is a difficult problem, or why DNS still is notorious for causing so much pain and suffering. Compared to the advancements in hardware and graphics and so many other areas.

There are resolvers for not just DNS but for users and other lookups. The list of resolvers is dynamic, they are configured in /etc/nsswitch.conf. The /etc/hosts lookup is part of the system. Where do the resolvers come from? It needs to be possible to install resolvers separately and dynamically load them. Unless you want to have NIS always installed. Better to install LDAP for those who need it.

It could be handled by a system daemon instead. We don't really need this in-process all the time.

Re: The state of binary compatibility on Linux and how to address it

#89

Post by AshamedCaptain »

terinjokes wrote: post #57
evidencetamper wrote: post #34

Earlier quoted context omitted.

As long as the library is available. Neither static nor dynamic linking is looking to solve the 20 year old binaries issue, so both will have different issues. But I think it's easier for me to find a 20 year old ISO of a Red Hat/Slackware where I can simply run the statically linked binary. Dependency hell for older distros become really difficult when the older packages are not archived anywhere anymore.

I've recently had to do this (to bisect when a change introduced a superficial bug into a 20-year-old program). I think "simply run" is viewing Linux of that era through rose-tinted glasses. Even for simple 2D "Super VGA" you're needing to choose the correct XFree86 implementation and still tweak your Xorg configuration. The emulated hardware also has bugs, since most of the focus is now on virtio drivers. (The 20-ye…

Plus there are other advantages that I always say should never be discounted, such as desktop integration, that you get when you can replace the toolkit (dynamic) libraries rather than just "running everything in a VM". Running everything in a VM is a poor excuse for backwards compatibility.

Re: The state of binary compatibility on Linux and how to address it

#90

Post by rollcat »

inftech wrote: post #86

I think the problem is that people treat Linux as OS instead of just a kernel. You should assume every Linux distro is a different OS so when you are shipping your app for Linux you're actually shipping your app for Debian, Fedora, Ubuntu etc

That's the sane way to tackle it. If you're the vendor, just target the top N (whatever value of N you can cope with).

I don't mean disrespect towards people running Alpine (hi), Arch, or Gentoo, but you wouldn't be running these distros if you aren't ready to handle their quirks.

TFA mostly talks about binary compat. Even if you can get away with statically linking everything, you still have to cope with the mess that is userspace fragmentation: https://tailscale.com/blog/sisyphean-dns-client-linux>

So yeah, supporting the top N gets you approximately sqrt(N/(N+1))% of the way. (Assuming desktop Linux market share is about 1%.)

Post reply on HN