Live data from Hacker News

The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

github.com

151–160 of 215 posts

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#151
post #45

Earlier quoted context omitted.

Justine identifies as a woman.

"identifies as" is an unnecessarily dismissive choice of words. She is a woman.

My statement was a fact, and in my opinion not politically loaded, yet respectful to Justine. I chose my words carefully.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#152
post #106

Earlier quoted context omitted.

Isn't the sole reason why linux sucks(sucked?) for games and other software exactly that there is a gazillion of different libraries with different versions, so you have zero assumptions about the state of the OS, which makes making sw for it such a pain?

Yes. Premature optimisation when it comes to dynamic linking is the reason for why the year of the Linux desktop is far away in my opinion

I just remember Jonathan Blow mentioning this in one of his streams.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#153
post #39

Earlier quoted context omitted.

they were prominent in game hacking 2005ish windows made hooking into game code much easier than before

Aren't all DLLs on the Windows platform compiled with an unusual instruction at the start of each function? This makes it possible to somehow hot patch the DLL after it is already in memory

I believe you're thinking of the x86 Hotpatching hook[1], which doesn't exist on x86-64[2] (in the same form, it uses a x86-64 safe one).

[1] https://devblogs.microsoft.com/oldnewthing/20110921-00/?p=95...

[2] https://devblogs.microsoft.com/oldnewthing/20221109-00/?p=10...

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#154
post #60

So what we need is essentially a "libc virtualization". But Musl is only available on Linux, isn't it? Cosmopolitan ( https://github.com/jart/cosmopolitan ) goes further and is available also on Mac and Windows, and it uses e.g. SIMD and other performance related improvements. Unfortunately, one has to cut through the marketing "magic" to find the main engineering value; stripping away the "polyglot" shell-script hac…

I find it amazing how much the mess that building C/C++ code has been for so many decades seems to have influenced the direction technology, the economy and even politics has been going. Really, what would the world look like if this problem had been properly solved? Would the centralization and monetization of the Internet have followed the same path? Would Windows be so dominant? Would social media have evolved to…

What I find amazing is why people continously claim glibc is the problem here. I have a commercial software binary from 1996 that _still works_ to this day. It even links with X11, and works under Xwayland.

The trick? It's not statically linked, but dynamically linked. And it doesn't like with anything other than glibc, X11 ... and bdb.

At this point I think people just do not know how binary compatibility works at all. Or they refer to a different problem that I am not familiar with.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#155

It's funny how people insist on wanting to link everything statically when shared libraries were specifically designed to have a better alternative. Even worse is containers, which has the disadvantage of both.

Dynamic libraries make a lot of sense as operating system interface when they guarantee a stable API and ABI (see Windows for how to do that) - the other scenarios where DLLs make sense is for plugin systems. But that's pretty much it, for anything else static linking is superior because it doesn't present an optimization barrier (especially for dead code elimination). No idea why the glibc can't provide API+ABI stab…

> (e.g. not being able to run an executable that was created on a more recent Linux system on an older Linux system even when the program doesn't access any new glibc entry points - the usually adviced solution is to link with an older glibc version, but that's also not trivial, unless you use the Zig toolchain).

In the era of containers, I do not understand why this is "Not trivial". I could do it with even a chroot.

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#156

I've been statically linking Nim binaries with musl. It's fantastic. Relatively easy to set up (just a few compiler flags and the musl toolchain), and I get an optimized binary that is indistinguishable from any other static C Linux binary. It runs on any machine we throw it at. For a newer-generation systems language, that is a massive selling point.

Yeah. I've been doing this for almost 10 years now. It's not APE/cosmopolitan (which also "kinda works" with Nim but has many lowest common denominator platform support issues, e.g. posix_fallocate). However, it does let you have very cross-Linux portable binaries. Maybe beyond Linux.

Some might appreciate a concrete instance of this advice inline here. For `foo.nim`, you can just add a `foo.nim.cfg`:

    @if gcc:
      gcc.exe       = "musl-gcc"
      gcc.linkerexe = "musl-gcc"
      passL         = "-static -s" @end
There is also a "NimScript" syntax you could use a `foo.nims`:

    if defined gcc:  # nim.cfg runs faster than NimScript
      switch "gcc.exe"      , "musl-gcc"
      switch "gcc.linkerexe", "musl-gcc"
      switch "passL"        , "-static -s"

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#157

Do I get this right that this effectively dlopens glibc (indirectly) into an executable that is statically linked to musl? How can the two runtimes coexist? What about malloc/free? AFAIK both libc's allocators take ownership of brk, that can't be good. What about malloc/free across the dynamic library interface? There are certainly libraries that hand out allocated objects and expect the user to free them, but that's…

You have to tell musl to use mmap instead of brk. You're right that it doesn't work in all cases but as long as you switch TLS on calls (and callbacks), at least with a project the size of Godot, you can approach a workable solution.

> You have to tell musl to use mmap instead of brk.

How do I do that? Is there a documented configuration of musl's allocator?

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#158

Earlier quoted context omitted.

I find it amazing how much the mess that building C/C++ code has been for so many decades seems to have influenced the direction technology, the economy and even politics has been going. Really, what would the world look like if this problem had been properly solved? Would the centralization and monetization of the Internet have followed the same path? Would Windows be so dominant? Would social media have evolved to…

What I find amazing is why people continously claim glibc is the problem here. I have a commercial software binary from 1996 that _still works_ to this day. It even links with X11, and works under Xwayland. The trick? It's not statically linked, but dynamically linked. And it doesn't like with anything other than glibc, X11 ... and bdb. At this point I think people just do not know how binary compatibility works at a…

We (small HPC system) just upgraded our OS from RHEL 7 to RHEL 9. Most user apps are dynamically linked, too.

You don't want to believe how many old binaries broke. Lot of ABI upgrades like libpng, ncurses, heck even stuff like readline and libtiff all changed just enough for linker errors to occur.

Ironically all the statically compiled stuff was fine. Some small things like you mention only linking to glibc and X11 was fine too. Funnily enough grabbing some old .so files from the RHEL 7 install and dumping them into LD_LIBRARY_PATH also worked better than expected.

But yeah, now that I'm writing this out, glibc was never the problem in terms of forwards compatibility. Now running stuff compiled on modern Ubuntu or RHEL 10 on the older OS, now that's a whole different story...

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#159

Earlier quoted context omitted.

I find it amazing how much the mess that building C/C++ code has been for so many decades seems to have influenced the direction technology, the economy and even politics has been going. Really, what would the world look like if this problem had been properly solved? Would the centralization and monetization of the Internet have followed the same path? Would Windows be so dominant? Would social media have evolved to…

How does this technical issue affect the economy and politics? In what way would the world be different just because we used a better linker?

existential crisis: so hot right now

Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen

#160

Earlier quoted context omitted.

What I find amazing is why people continously claim glibc is the problem here. I have a commercial software binary from 1996 that _still works_ to this day. It even links with X11, and works under Xwayland. The trick? It's not statically linked, but dynamically linked. And it doesn't like with anything other than glibc, X11 ... and bdb. At this point I think people just do not know how binary compatibility works at a…

We (small HPC system) just upgraded our OS from RHEL 7 to RHEL 9. Most user apps are dynamically linked, too. You don't want to believe how many old binaries broke. Lot of ABI upgrades like libpng, ncurses, heck even stuff like readline and libtiff all changed just enough for linker errors to occur. Ironically all the statically compiled stuff was fine. Some small things like you mention only linking to glibc and X11…

> Funnily enough grabbing some old .so files from the RHEL 7 install and dumping them into LD_LIBRARY_PATH also worked better than expected.

Why "better than expected"? I can run the entire userspace from Debian Etch on a kernel built two days ago... some kernel settings need to be changed (because of the old glibc! but it's not glibc's fault: it's the kernel who broke things), but it works.

> Now running stuff compiled on modern Ubuntu or RHEL 10 on the older OS, now that's a whole different story...

But this is a different problem, and no one makes promises here (not the kernel, not musl). So all the talk of statically linking with musl to get such type of compatibility is bullshit (at some point, you're going to hit a syscall/instruction/whatever that the newer musl does that the older kernel/hardware does not support).

Post reply on HN