Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The .a file is a relic: Why static archives were a bad idea all along

medium.com

Post reply on HN
Print view View original

101–110 of 136 posts

Re: The .a file is a relic: Why static archives were a bad idea all along

#101

Post by viraptor »

TuxSH wrote: post #11

> This design decision at the source level, means that in our linked binary we might not have the logic for the 3DES building block, but we would still have unused decryption functions for AES256. Do people really not know about `-ffunction-sections -fdata-sections` & `-Wl,--gc-sections` (doesn't require LTO)? Why is it used so little when doing statically-linked builds? > Let’s say someone in our library designed th…

https://xkcd.com/2501/

... It's easy to forget that the average person probably only knows two or three linker flags ...

Re: The .a file is a relic: Why static archives were a bad idea all along

#102

Post by lokar »

layer8 wrote: post #94
lokar wrote: post #84

Earlier quoted context omitted.

The call to init should be idempotent

That can be difficult in a multi-threaded environment with dynamically loaded shared libraries. Or at least it isn’t something that’s generally expected to be guaranteed to work.

Ideally they would do the explicit init at startup before starting threads

Re: The .a file is a relic: Why static archives were a bad idea all along

#103

Post by TuxSH »

layer8 wrote: post #94
lokar wrote: post #84

Earlier quoted context omitted.

The call to init should be idempotent

That can be difficult in a multi-threaded environment with dynamically loaded shared libraries. Or at least it isn’t something that’s generally expected to be guaranteed to work.

C++ "magic statics" handle that use case (but with hidden atomic flag load (& more) costs at each access)

Re: The .a file is a relic: Why static archives were a bad idea all along

#104

Post by Orphis »

benreesman wrote: post #92
Orphis wrote: post #72

Earlier quoted context omitted.

pkg-config works great in limited scenarios. If you try to do anything more complex, you'll probably run into some complex issues that require modifying the supplied .pc files from your vendor. There's is a new standard that is being developed by some industry experts that is aiming to address this called CPS. You can read the documentation on the website: https://cps-org.github.io/cps/ . There's a section with some…

`pkg-config` works great in just about any standard scenario: it puts flags on a compile and link line that have been understood by every C compiler and linker since the 1970s. Here's Bazel consuming it with zero problems, and if you have a nastier problem than a low-latency network system calling `liburing` on specific versions of the kernel built with Bazel? Stop playing. The last thing we need is another failed st…

Have you read the rationale about CPS? It gives clear examples as to why it doesn't work. You need to parse the files and then parse all the compiler and linker arguments in order to understand what to do with those to properly consume them.

What do you do if you use a compiler or linker that doesn't use the same command line parameters as they are written in the pc file? What do you do when different packages you depend on have conflicting options, for example one depending against different C or C++ language versions?

It's fine in a limited and closed environment, it does not work for proper distribution, and your Bazel rules prove it as it is not working in all environments clearly. It does not work with MSVC style flags, or handles include files well (hh, hxx...). Not saying it can't be fixed, but that's just a very limited integration, which proves the point of having a better format for tool consumption.

And you're not the only one who has worked in a FAANG company around and dealt with large and complex build graphs. But for the most part, FAANGs don't all care about consuming pkg-config files, most will just rewrite the build files for Blaze / Bazel (or Buck2 from what I've heard). Very few people want to consume binary archives as you can't rebuild with the new flavor of the week toolchain and use new compiler optimizations, or proper LTO etc.

Re: The .a file is a relic: Why static archives were a bad idea all along

#105

Post by high_na_euv »

trinix912 wrote: post #51
high_na_euv wrote: post #48

Earlier quoted context omitted.

>The fact you (probably) never had to manipulate it directly just mean your higher level work never brought you deep enough where it starts to be a problem. Did you just agree with me that other prog. ecosystems solved the building system challenge?

They solved it by building on top of what C (or LLVM, to be precise) does, not by avoiding or replacing it. What should C do to solve it? Add another layer of abstraction on top of it? CMake does that and people complain about the extra complexity.

Bro, cmake is garbage. Having to debug cmske scripts is pain

Also how e.g dotnet builds on top of c or llvm?

Re: The .a file is a relic: Why static archives were a bad idea all along

#106

Post by tester756 »

uecker wrote: post #88
adev_ wrote: post #47

Earlier quoted context omitted.

> Why things that are solved in other programming ecosystems are impossible in c cpp world, like sane building system This is such an ignorant comment. Most other natively compiled languages have exactly the same concept behind: Object files, Shared Libraries, collection of object and some kind of configuration description of the compilation pipeline. Even high level languages like Rust has that (to some extend). The…

I think people today often do not understand any more that C like many other things in the UNIX world is a tool, not a complete framework. But somehow people expect a complete convenient framework with batteries included. They see it has a deficiency that C by itself does not provide many things. I see it as one of its major strengths and this is one of the reasons why I prefer it.

Sane building system is pretty much basic thing in modern language

You'd expect something as mature as C to have such a thing by default

Re: The .a file is a relic: Why static archives were a bad idea all along

#107

Post by immibis »

rixed wrote: post #33

Do people who write this kind of pieces with such peremptory titles really believe that they finally came about to understand everything better after decades of ignorance? Chesterton’s Fence yada yada?

Linking works the way it does because of inertia.

There's a pretty broad space of possible ways to link (both static and dynamic). Someone once wrote one of them, and it was good enough, so it stuck, and spread, and now everything assumes it. It's far from the only possible way, but you'd have to write new tooling to do it a different way, and that's rarely worth it. The way they selected is pretty reasonable, but not optimal for some use cases, and it may even be pathological for some.

At least Linux provides the necessary hooks to change it: your PT_INTERP doesn't have to be /lib64/ld-linux-x86-64.so.2

There are many different ways linking can work (both static and dynamic); the currently selected ways are pretty reasonable points in that space, but not the only ones, and can even be pathological in some scenarios.

Actually, a whole lot of things in computing work this way, from binary numbers, to 8-bit bytes, to filesystems, to file handles as a concept, to IP addresses and ports, to raster displays. There were many solutions to a problem, and then one was implemented, and it worked pretty well and it spread, even though other solutions were also possible, and now we can build on top of that one instead of worrying about which one to choose underneath. If you wanted to make a computer from scratch you'd have to decide whether binary is better than decimal, bi-quinary or balanced ternary... or just copy the safe, widespread option. (Contrary to popular belief, very early computers used a variety of number bases other than binary)

Re: The .a file is a relic: Why static archives were a bad idea all along

#108

Post by bsder »

asveikau wrote: post #65
pjmlp wrote: post #42

Earlier quoted context omitted.

How can they be expected to learn this, when it is now fashionable to treat C and C++ as if they are scripting languages, shipping header only files? We already had scripting engines for those languages in the 1990's, and the fact they are hardly available nowadays kind of tells of their commercial success, with exception of ROOT.

It makes more sense for c++ due to templates, but the header only C library trend is indeed very strange. It's not surprising that people are coming up now who are writing articles about being confused by static linking behavior.

> It makes more sense for c++ due to templates, but the header only C library trend is indeed very strange.

This is a symptom of the build tools ecosystem for C and C++ being an absolute dumpster fire (looking at you CMake).

Re: The .a file is a relic: Why static archives were a bad idea all along

#109

Post by izacus »

greenavocado wrote: post #61
TuxSH wrote: post #11

> This design decision at the source level, means that in our linked binary we might not have the logic for the 3DES building block, but we would still have unused decryption functions for AES256. Do people really not know about `-ffunction-sections -fdata-sections` & `-Wl,--gc-sections` (doesn't require LTO)? Why is it used so little when doing statically-linked builds? > Let’s say someone in our library designed th…

> Do people really not know about $OBSCURE_GCC_FLAG? Do you know what you sound like?

And do you know how you sound like when you call a well known, first hit on Google/AI, set of flags as obscure?

Re: The .a file is a relic: Why static archives were a bad idea all along

#110

Post by dwattttt »

layer8 wrote: post #56

> Something like a “Static Bundle Object” (.sbo) file, that will be closer to a Shared Object (.so) file, than to the existing Static Archive (.a) file. Is there something missing from .so files that wouldn’t allow them to be used as a basis for static linking? Ideally, you’d only distribute one version of the library that third parties can decide to either link statically or dynamically.

Shared libraries are linked together in a lossy step. I don't believe it's theoretically impossible; as an unsatisfying proof of concept, you could 'statically' link the .so by archiving it in the final binary, unpacking it at runtime, and dynamically linking it.

The static linker would be prevented from seeing multiple copies of code too.

Post reply on HN