Live data from Hacker News

Comparison of C/POSIX standard library implementations for Linux

etalabs.net

21–30 of 58 posts

Re: Comparison of C/POSIX standard library implementations for Linux

#21

Fun libc comparison by the author of musl. My getaway is: glibc is bloated but fast. Quite unexpected combination. Am I right?

Not quite correct. glibc is slow if you need to be able to fork quickly.

However, it does have super-optimized string/memory functions. There are highly optimized assembly language implementations of them that use SIMD for dozens of different CPUs.

Re: Comparison of C/POSIX standard library implementations for Linux

#22

It really ought to lead with the license of each library. I was considering dietlibc until I got to the bottom - GPLv2. I am a GPL apologist and even I can appreciate that this is a nonstarter; even GNU's libc is only LGPL!

musl seems to have displaced dietLibc. Much more complete yet fairly small and light.

Note that dietlibc is the project of a sole coder in the CCC sphere from Berlin (Fefe). His main objective was to learn how low level infra is implemented and started using it in some of his other projects after realizing that there is a lot of bloat he can skip with just implementing the bare essentials. Musl has a different set of objectives.

Re: Comparison of C/POSIX standard library implementations for Linux

#23

That table is unfortunately quite old. I can't personally say what have changed, but it is hard to put much confidence in the relevance of the information.

Yeah, also it doesn't compare actual implementations, just plain checkboxes. I'm aware of two specific substantial performance regressions for musl: exact floating point printing (it uses Dragon4 but implemented it way slower than it could have been) and memory allocator (for a long time it didn't any sort of arena like pretty much every modern allocator---now it does with mallocng though).

Re: Comparison of C/POSIX standard library implementations for Linux

#24

Earlier quoted context omitted.

It's called glibc. Essentially all that "bloat" is conditionally compiled, if your target isn't an ancient or bizarre platform it won't get included in the runtime.

That’s mostly true, but not quite. For instance, suppose you aim to support all of 32/64-bit and little/big-endian. You’ll likely end up factoring straightforward math operations out into standalone functions. Granted, those will probably get inlined, but it may mean your structure is more abstracted than it would be otherwise. Just supporting the options has implications. That’s not the strongest example. I just mea…

The way glibc's source works (for something like math functions) is that essentially every function is implemented in its own file, and various config knobs can provide extra directories to compile and provide function definitions. This can make actually finding the implementation that's going to be used difficult, since a naive search for the function name can turn up like 20 different function definitions, and working out which one is actually in play can be difficult (especially since it's more than just the architecture name).

Math functions aren't going to be strongly impacted by diverse hardware support. In practice, you largely care about 32-bit and 64-bit IEEE 754 types, which means your macros to decompose floating-point types to their constituent sign/exponent/significand fields are already going to be pretty portable even across different endianness (just bitcast to a uint32_t/uint64_t, and all of the shift logic will remain the same). And there's not much reason to vary the implementation except to take advantage of hardware instructions that implement the math functions directly... which are generally better handled by the compiler anyways.

Re: Comparison of C/POSIX standard library implementations for Linux

#25

My own perf comparison: when I switched from Fil-C running on my system’s libc (recent glibc) for yololand to my own build of musl, I got a 1-2% perf regression. My best guess is that it’s because glibc’s memcpy/memmove/memset are better. Couldn’t have been the allocator since Fil-C’s runtime has its own allocator.

Interesting! Will you stick around with the musl build? And if so, why?

Not sure but in likely to because right now I to use the same libc in userland (the Fil-C compiled part) and yololand (the part compiled by normal C that is below the runtime) and the userland libc is musl.

Having them be the same means that if there is any libc function that is best implemented by having userland call a Fil-C runtime wrapper for the yololand implementation (say because what it’s doing requires platform specific assembly) then I can be sure that the yololand libc really implements that function the same way with all the same corner cases.

But there aren’t many cases of that and they’re hacks that I might someday remove. So I probably won’t have this “libc sandwich” forever

Re: Comparison of C/POSIX standard library implementations for Linux

#26

Fun libc comparison by the author of musl. My getaway is: glibc is bloated but fast. Quite unexpected combination. Am I right?

A lot of the “slowness” of MUSL is the default allocator. It can be swapped out. For example, Chimera Linux uses MUSL with mimalloc and it is quite snappy.

That's a great combo. I like LLVM libc in overlay mode with musl beneath and mimalloc. Performance is excellent.

Re: Comparison of C/POSIX standard library implementations for Linux

#27

My own perf comparison: when I switched from Fil-C running on my system’s libc (recent glibc) for yololand to my own build of musl, I got a 1-2% perf regression. My best guess is that it’s because glibc’s memcpy/memmove/memset are better. Couldn’t have been the allocator since Fil-C’s runtime has its own allocator.

When I was working with Envoy Proxy, it was known that perf was worse with musl than with glibc. We went through silly hoops to have a glibc Envoy running in an Alpine (musl) container.

Re: Comparison of C/POSIX standard library implementations for Linux

#28

My own perf comparison: when I switched from Fil-C running on my system’s libc (recent glibc) for yololand to my own build of musl, I got a 1-2% perf regression. My best guess is that it’s because glibc’s memcpy/memmove/memset are better. Couldn’t have been the allocator since Fil-C’s runtime has its own allocator.

What's Fil-C? Okay, found it myself, looks cool: https://github.com/pizlonator/llvm-project-deluge/

What's yoyoland? All I can find is an amusement park in Bangkok, and some 1990s-era communication software for Classic Mac OS: https://www.macintoshrepository.org/39495-yoyo-2-1

Re: Comparison of C/POSIX standard library implementations for Linux

#29

My own perf comparison: when I switched from Fil-C running on my system’s libc (recent glibc) for yololand to my own build of musl, I got a 1-2% perf regression. My best guess is that it’s because glibc’s memcpy/memmove/memset are better. Couldn’t have been the allocator since Fil-C’s runtime has its own allocator.

What's Fil-C? Okay, found it myself, looks cool: https://github.com/pizlonator/llvm-project-deluge/ What's yoyoland? All I can find is an amusement park in Bangkok, and some 1990s-era communication software for Classic Mac OS: https://www.macintoshrepository.org/39495-yoyo-2-1

The Fil-C stack is composed of :

- Userland: the place where you C code lives. Like the normal userland you're familiar with, but everything is compiled with Fil-C, so it's memory safe.

- Yololand: the place where Fil-C's runtime lives. Fil-C's runtime is about 100,000 lines of C code (almost entirely written by me), which currently has libc as a dependency (because the runtime makes syscalls using the normal C functions for syscalls rather than using assembly directly; also the runtime relies on a handful of libc utility functions that aren't syscalls, like memcpy).

So Fil-C has two libc's. The yololand libc (compiled with a normal C compiler, only there to support the runtime) and the userland libc (compiled with the Fil-C compiler like everything else in Fil-C userland, and this is what your C code calls into).

Re: Comparison of C/POSIX standard library implementations for Linux

#30
post #22

Earlier quoted context omitted.

musl seems to have displaced dietLibc. Much more complete yet fairly small and light.

Note that dietlibc is the project of a sole coder in the CCC sphere from Berlin (Fefe). His main objective was to learn how low level infra is implemented and started using it in some of his other projects after realizing that there is a lot of bloat he can skip with just implementing the bare essentials. Musl has a different set of objectives.

I follow diet but it is definitely not ready for general use like musl and probably never will be. There aren't a lot of eyeballs on it.
Post reply on HN