Live data from Hacker News

Show HN: A 166 KB file for cross compiling glibc for any version, any target

github.com

1–10 of 34 posts

Re: Show HN: A 166 KB file for cross compiling glibc for any version, any target

#2
Some context:

In master branch of Zig right now (before https://github.com/ziglang/zig/pull/10330 is merged), status quo:

    $ cat ../lib/libc/glibc/*.txt | wc -c
    205219
    $ cat ../lib/libc/glibc/*.txt | xz | wc -c
    22976
Zig supports targeting every version of glibc for any target architecture. The information required to do this takes up 200 KB installation size / 22 KB tarball size, and it has the following problems (which are causing several bugs):

* fails to represent symbols that migrate from one library to another between glibc versions

* fails to represent a distinction between functions and objects

* fails to represent object sizes

With this new glibc-abi-tool:

    $ cat abilists | wc -c
    169421
    $ cat abilists | xz | wc -c
    24880
165 KB installation size / 24 KB tarball size, and all the above issues are solved. It also should be slightly faster to load/parse for the compiler since it is fewer KB to load from disk.

The code that uses this file is here: https://github.com/ziglang/zig/blob/0d7331b6f6af85bd45829514... It creates c.s, pthread.s, dl.s, etc., which are assembled into libc.so, libpthread.so, libdl.so, etc., which are placed on the linker line when cross compiling. Thanks to this improved dataset, they can now accurately model the glibc that will be on any CPU architecture, any glibc version.

If I just naively shipped every version of the .abilist files:

    $ cat (find glibc/ -name "*.abilist") | wc -c
    37041906
    $ cat (find glibc/ -name "*.abilist") | xz | wc -c
    205036
...it would be 35 MiB installation size, 200 KB tarball size. So I have effectively achieved a compression ratio of 219:1 by implementing a bespoke encoding of this information.

Re: Show HN: A 166 KB file for cross compiling glibc for any version, any target

#3
What does "targeting any glibc version" mean? Is it the solution to the problem where you have to compile on an ancient Debian version to produce a binary that works on all Linux distributions because it otherwise links to too-new symbol versions in glibc?

Can I use that outside of Zig, for compiling C++ code?

Re: Show HN: A 166 KB file for cross compiling glibc for any version, any target

#4

Some context: In master branch of Zig right now (before https://github.com/ziglang/zig/pull/10330 is merged), status quo: $ cat ../lib/libc/glibc/*.txt | wc -c 205219 $ cat ../lib/libc/glibc/*.txt | xz | wc -c 22976 Zig supports targeting every version of glibc for any target architecture. The information required to do this takes up 200 KB installation size / 22 KB tarball size, and it has the following problems (wh…

Totally off-topic: I installed Zig on Saturday afternoon and by the evening I had my first program do some useful stuff, on Sunday I added some C code, then added some tests and by Sunday evening I had a pure Zig project again (and afterwards learned that translate-to-zig is a thing). I gotta say I'm very impressed with Zig and partial evaluation is just such a nice, mentally simple technique for generics and lots of other things. The syntax is pretty nice, too, though the prefix-[] are kinda weird (just way too used to C/C++'s postfix array notation) and ranged loops are somewhat awkward. The @ naming for builtins confused me for a bit because I thought the @ itself was syntax. And array literals are kinda weird sometimes though I've seen that apparently in 0.9 "&[_][]const u8{ "-std=c99", "-O3" }" can just be written as ".{ "-std=c99", "-O3" }". And I was kinda surprised about array types in function signatures being implicitly const, I think they're like that to make array types behave mostly like value types while only passing a pointer?

Re: Show HN: A 166 KB file for cross compiling glibc for any version, any target

#5

What does "targeting any glibc version" mean? Is it the solution to the problem where you have to compile on an ancient Debian version to produce a binary that works on all Linux distributions because it otherwise links to too-new symbol versions in glibc? Can I use that outside of Zig, for compiling C++ code?

zig also functions as a c and c++ compiler via 'zig cc'.

Re: Show HN: A 166 KB file for cross compiling glibc for any version, any target

#6

What does "targeting any glibc version" mean? Is it the solution to the problem where you have to compile on an ancient Debian version to produce a binary that works on all Linux distributions because it otherwise links to too-new symbol versions in glibc? Can I use that outside of Zig, for compiling C++ code?

The correct solution for that is to just cross compile to whatever older CPU you want and target a sysroot that has a base image of whatever older glibc you wanted: you don't need to actually compile on the older device...

Re: Show HN: A 166 KB file for cross compiling glibc for any version, any target

#8

Some context: In master branch of Zig right now (before https://github.com/ziglang/zig/pull/10330 is merged), status quo: $ cat ../lib/libc/glibc/*.txt | wc -c 205219 $ cat ../lib/libc/glibc/*.txt | xz | wc -c 22976 Zig supports targeting every version of glibc for any target architecture. The information required to do this takes up 200 KB installation size / 22 KB tarball size, and it has the following problems (wh…

Totally off-topic: I installed Zig on Saturday afternoon and by the evening I had my first program do some useful stuff, on Sunday I added some C code, then added some tests and by Sunday evening I had a pure Zig project again (and afterwards learned that translate-to-zig is a thing). I gotta say I'm very impressed with Zig and partial evaluation is just such a nice, mentally simple technique for generics and lots of…

> the prefix-[] are kinda weird (just way too used to C/C++'s postfix array notation)

I think over time the C/C++ way is going to go the way of the dodo; IIRC, rust and go both use the prefix-[]. It creates a clear, unambiguous way of defining the type (no "spiral typing" jokes for the modern languages).

Re: Show HN: A 166 KB file for cross compiling glibc for any version, any target

#9
post #8

Earlier quoted context omitted.

Totally off-topic: I installed Zig on Saturday afternoon and by the evening I had my first program do some useful stuff, on Sunday I added some C code, then added some tests and by Sunday evening I had a pure Zig project again (and afterwards learned that translate-to-zig is a thing). I gotta say I'm very impressed with Zig and partial evaluation is just such a nice, mentally simple technique for generics and lots of…

> the prefix-[] are kinda weird (just way too used to C/C++'s postfix array notation) I think over time the C/C++ way is going to go the way of the dodo; IIRC, rust and go both use the prefix-[]. It creates a clear, unambiguous way of defining the type (no "spiral typing" jokes for the modern languages).

I wish Rust used prefix &[]i32 for a slice, and [5][2]i32 for a 5x2 array. Instead it uses &[i32] for a slice, and [[i32; 2]; 5] for a 5x2 array. Reading types backwards is alive and well.

Re: Show HN: A 166 KB file for cross compiling glibc for any version, any target

#10

Some context: In master branch of Zig right now (before https://github.com/ziglang/zig/pull/10330 is merged), status quo: $ cat ../lib/libc/glibc/*.txt | wc -c 205219 $ cat ../lib/libc/glibc/*.txt | xz | wc -c 22976 Zig supports targeting every version of glibc for any target architecture. The information required to do this takes up 200 KB installation size / 22 KB tarball size, and it has the following problems (wh…

How much work is it to then maintain this bespoke encoding? Without having much context on it, it sounds similar to having to maintain unicode identifier character tables for unicode-aware parsers (e.g. Javascript's). Updating unicode tables after unicode version releases can be done w/ somewhat hack-ish scripting of parsing txt spec files but it's admittedly not super clean, I'm curious what's involved when dealing w/ abi lists over time.
Post reply on HN