Earlier quoted context omitted.
Justine identifies as a woman.
"identifies as" is an unnecessarily dismissive choice of words. She is a woman.
The Holy Grail of Linux Binary Compatibility: Musl and Dlopen
151–160 of 215 posts
Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen
#152Earlier quoted context omitted.
Isn't the sole reason why linux sucks(sucked?) for games and other software exactly that there is a gazillion of different libraries with different versions, so you have zero assumptions about the state of the OS, which makes making sw for it such a pain?
Yes. Premature optimisation when it comes to dynamic linking is the reason for why the year of the Linux desktop is far away in my opinion
Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen
#153Earlier quoted context omitted.
they were prominent in game hacking 2005ish windows made hooking into game code much easier than before
Aren't all DLLs on the Windows platform compiled with an unusual instruction at the start of each function? This makes it possible to somehow hot patch the DLL after it is already in memory
[1] https://devblogs.microsoft.com/oldnewthing/20110921-00/?p=95...
[2] https://devblogs.microsoft.com/oldnewthing/20221109-00/?p=10...
Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen
#154So what we need is essentially a "libc virtualization". But Musl is only available on Linux, isn't it? Cosmopolitan ( https://github.com/jart/cosmopolitan ) goes further and is available also on Mac and Windows, and it uses e.g. SIMD and other performance related improvements. Unfortunately, one has to cut through the marketing "magic" to find the main engineering value; stripping away the "polyglot" shell-script hac…
I find it amazing how much the mess that building C/C++ code has been for so many decades seems to have influenced the direction technology, the economy and even politics has been going. Really, what would the world look like if this problem had been properly solved? Would the centralization and monetization of the Internet have followed the same path? Would Windows be so dominant? Would social media have evolved to…
The trick? It's not statically linked, but dynamically linked. And it doesn't like with anything other than glibc, X11 ... and bdb.
At this point I think people just do not know how binary compatibility works at all. Or they refer to a different problem that I am not familiar with.
Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen
#155It's funny how people insist on wanting to link everything statically when shared libraries were specifically designed to have a better alternative. Even worse is containers, which has the disadvantage of both.
Dynamic libraries make a lot of sense as operating system interface when they guarantee a stable API and ABI (see Windows for how to do that) - the other scenarios where DLLs make sense is for plugin systems. But that's pretty much it, for anything else static linking is superior because it doesn't present an optimization barrier (especially for dead code elimination). No idea why the glibc can't provide API+ABI stab…
In the era of containers, I do not understand why this is "Not trivial". I could do it with even a chroot.
Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen
#156I've been statically linking Nim binaries with musl. It's fantastic. Relatively easy to set up (just a few compiler flags and the musl toolchain), and I get an optimized binary that is indistinguishable from any other static C Linux binary. It runs on any machine we throw it at. For a newer-generation systems language, that is a massive selling point.
Some might appreciate a concrete instance of this advice inline here. For `foo.nim`, you can just add a `foo.nim.cfg`:
@if gcc:
gcc.exe = "musl-gcc"
gcc.linkerexe = "musl-gcc"
passL = "-static -s" @end
There is also a "NimScript" syntax you could use a `foo.nims`: if defined gcc: # nim.cfg runs faster than NimScript
switch "gcc.exe" , "musl-gcc"
switch "gcc.linkerexe", "musl-gcc"
switch "passL" , "-static -s"Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen
#157Do I get this right that this effectively dlopens glibc (indirectly) into an executable that is statically linked to musl? How can the two runtimes coexist? What about malloc/free? AFAIK both libc's allocators take ownership of brk, that can't be good. What about malloc/free across the dynamic library interface? There are certainly libraries that hand out allocated objects and expect the user to free them, but that's…
You have to tell musl to use mmap instead of brk. You're right that it doesn't work in all cases but as long as you switch TLS on calls (and callbacks), at least with a project the size of Godot, you can approach a workable solution.
How do I do that? Is there a documented configuration of musl's allocator?
Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen
#158Earlier quoted context omitted.
I find it amazing how much the mess that building C/C++ code has been for so many decades seems to have influenced the direction technology, the economy and even politics has been going. Really, what would the world look like if this problem had been properly solved? Would the centralization and monetization of the Internet have followed the same path? Would Windows be so dominant? Would social media have evolved to…
What I find amazing is why people continously claim glibc is the problem here. I have a commercial software binary from 1996 that _still works_ to this day. It even links with X11, and works under Xwayland. The trick? It's not statically linked, but dynamically linked. And it doesn't like with anything other than glibc, X11 ... and bdb. At this point I think people just do not know how binary compatibility works at a…
You don't want to believe how many old binaries broke. Lot of ABI upgrades like libpng, ncurses, heck even stuff like readline and libtiff all changed just enough for linker errors to occur.
Ironically all the statically compiled stuff was fine. Some small things like you mention only linking to glibc and X11 was fine too. Funnily enough grabbing some old .so files from the RHEL 7 install and dumping them into LD_LIBRARY_PATH also worked better than expected.
But yeah, now that I'm writing this out, glibc was never the problem in terms of forwards compatibility. Now running stuff compiled on modern Ubuntu or RHEL 10 on the older OS, now that's a whole different story...
Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen
#159Earlier quoted context omitted.
I find it amazing how much the mess that building C/C++ code has been for so many decades seems to have influenced the direction technology, the economy and even politics has been going. Really, what would the world look like if this problem had been properly solved? Would the centralization and monetization of the Internet have followed the same path? Would Windows be so dominant? Would social media have evolved to…
How does this technical issue affect the economy and politics? In what way would the world be different just because we used a better linker?
Re: The Holy Grail of Linux Binary Compatibility: Musl and Dlopen
#160Earlier quoted context omitted.
What I find amazing is why people continously claim glibc is the problem here. I have a commercial software binary from 1996 that _still works_ to this day. It even links with X11, and works under Xwayland. The trick? It's not statically linked, but dynamically linked. And it doesn't like with anything other than glibc, X11 ... and bdb. At this point I think people just do not know how binary compatibility works at a…
We (small HPC system) just upgraded our OS from RHEL 7 to RHEL 9. Most user apps are dynamically linked, too. You don't want to believe how many old binaries broke. Lot of ABI upgrades like libpng, ncurses, heck even stuff like readline and libtiff all changed just enough for linker errors to occur. Ironically all the statically compiled stuff was fine. Some small things like you mention only linking to glibc and X11…
Why "better than expected"? I can run the entire userspace from Debian Etch on a kernel built two days ago... some kernel settings need to be changed (because of the old glibc! but it's not glibc's fault: it's the kernel who broke things), but it works.
> Now running stuff compiled on modern Ubuntu or RHEL 10 on the older OS, now that's a whole different story...
But this is a different problem, and no one makes promises here (not the kernel, not musl). So all the talk of statically linking with musl to get such type of compatibility is bullshit (at some point, you're going to hit a syscall/instruction/whatever that the newer musl does that the older kernel/hardware does not support).