The .a file is a relic: Why static archives were a bad idea all along • Hacker News Board

Re: The .a file is a relic: Why static archives were a bad idea all along

#91

Post by benreesman » Tue, Jul 22, 2025, 6:12 PM UTC

tux3 wrote: post #49

benreesman wrote: post #46

Earlier quoted context omitted.

`boost` is a little sticky too: https://gist.github.com/b7r6/e9d56c0f6d55bc0620b2ce190e15d44... but for your trouble: https://gist.github.com/b7r6/0cc4248e24288551bcc06281c831148... If there's interest in this I can make a priority out of trying to get it open-sourced.

Yes, boost is one of those that gave me the biggest trouble as well. I feel like we really need better toolchains in the first place. None of this intrinsically needs to be made complex, it's all a lack of proper support in the standard tools.

It's not though, as you can see from building two of the most notoriously nasty libraries on God's earth, emitting reasonable `pkg-config` is trivial, it's string concatenation.

The problem is misaligned incentives: CMake is bad, but it was sort of in the right place at the right time and became a semi-standard, and it's not in the interests of people who work in the CMake ecosystem to emit correct standard artifact manifests.

Dynamic linking by default is bad, but the gravy train on that runs from Docker to AWS to insecure-by-design TLS libraries.

The fix is for a few people who care more about good computing than money or fame to do simple shit like I'm doing above and make it available. CMake will be very useful in destroying CMake: it already encodes the same information that correct `pkg-config` needs.

Re: The .a file is a relic: Why static archives were a bad idea all along

#92

Post by benreesman » Tue, Jul 22, 2025, 6:19 PM UTC

Orphis wrote: post #72

benreesman wrote: post #8

It is unclear to me what the author's point is. Its seems to center on the example of DPDK being difficult to link (and it is a bear, I've done it recently). But its full of strawmen and falsehoods, the most notable being the claims about the deficienies of pkg-config. pkg-config works great, it is just very rarely produced correctly by CMake. I have tooling and a growing set of libraries that I'll probably open sour…

pkg-config works great in limited scenarios. If you try to do anything more complex, you'll probably run into some complex issues that require modifying the supplied .pc files from your vendor. There's is a new standard that is being developed by some industry experts that is aiming to address this called CPS. You can read the documentation on the website: https://cps-org.github.io/cps/ . There's a section with some…

`pkg-config` works great in just about any standard scenario: it puts flags on a compile and link line that have been understood by every C compiler and linker since the 1970s.

Here's Bazel consuming it with zero problems, and if you have a nastier problem than a low-latency network system calling `liburing` on specific versions of the kernel built with Bazel? Stop playing.

The last thing we need is another failed standard further balkanizing an ecosystem that has worked fine if used correctly for 40+ years. I don't know what industry expert means, but I've done polyglot distributed builds at FAANG scale for a living, so my appeal to authority is as good as anyone's and I say `pkg-config` as a base for the vast majority of use cases with some special path for like, compiling `nginx` with it's zany extension mechanism is just fine.

https://gist.github.com/b7r6/316d18949ad508e15243ed4aa98c80d...

Re: The .a file is a relic: Why static archives were a bad idea all along

#93

Post by benreesman » Tue, Jul 22, 2025, 6:25 PM UTC

KWxIUElW8Xt0tD9 wrote: post #57

benreesman wrote: post #8

It is unclear to me what the author's point is. Its seems to center on the example of DPDK being difficult to link (and it is a bear, I've done it recently). But its full of strawmen and falsehoods, the most notable being the claims about the deficienies of pkg-config. pkg-config works great, it is just very rarely produced correctly by CMake. I have tooling and a growing set of libraries that I'll probably open sour…

yes DLL hell is the issue with dynamic linking -- how many versions of given libraries are required for the various apps you want to install? -- and then you want to upgrade something and it requires yet another version of some library -- there is really no perfect solution to all this

You reconcile a library set for your application. That's happening whether you realize it or not, and whether you want to or not.

The question is, do you want it to happen under your control in an organized way that produces fast, secure, portable artifacts, or do you want it to happen in some random way controlled by other people at some later date that will probably break or be insecure or both.

There's an analogy here to systems like `pip` and systems with solvers in them like `uv`: yeah, sometimes you can call `pip` repeatedly and get something that runs in that directory on that day. And neat, if you only have to run it once, fine.

But if you ship that, you're externalizing the costs to someone else, which is a dick move. `uv` tells you on the spot that there's no solution, and so you have to bump a version bound to get a "works here and everywhere and pretty much forever" guarantee that's respectful of other people.

Re: The .a file is a relic: Why static archives were a bad idea all along

#94

Post by layer8 » Tue, Jul 22, 2025, 6:32 PM UTC

lokar wrote: post #84

layer8 wrote: post #82

Earlier quoted context omitted.

What if you use two libraries A and B that both happen to use library C under the hood? Is the application expected to initialize all dependencies in the right order at the top level? Or is library initialization supposed to be idempotent? This all works as long as libraries are “flat”, but doesn’t scale very well once libraries are built on top of each other and want to hide implementation details.

The call to init should be idempotent

That can be difficult in a multi-threaded environment with dynamically loaded shared libraries. Or at least it isn’t something that’s generally expected to be guaranteed to work.

Re: The .a file is a relic: Why static archives were a bad idea all along

#95

Post by eyalitki » Tue, Jul 22, 2025, 6:34 PM UTC

benreesman wrote: post #92

Orphis wrote: post #72

Earlier quoted context omitted.

pkg-config works great in limited scenarios. If you try to do anything more complex, you'll probably run into some complex issues that require modifying the supplied .pc files from your vendor. There's is a new standard that is being developed by some industry experts that is aiming to address this called CPS. You can read the documentation on the website: https://cps-org.github.io/cps/ . There's a section with some…

`pkg-config` works great in just about any standard scenario: it puts flags on a compile and link line that have been understood by every C compiler and linker since the 1970s. Here's Bazel consuming it with zero problems, and if you have a nastier problem than a low-latency network system calling `liburing` on specific versions of the kernel built with Bazel? Stop playing. The last thing we need is another failed st…

If someone needs a wrapper for a technology, that modifies the output it provides (like meson and bazel do), maybe there is an issue with said technology.

If pkg-config was never meant to be consumed directly, and was always meant to be post processed, then we are missing this post processing tool. Reinventing it in every compilation technology again and again is suboptimal, and at least Make and CMake do not have this post processing support.

Re: The .a file is a relic: Why static archives were a bad idea all along

#96

Post by triknomeister » Tue, Jul 22, 2025, 6:50 PM UTC

eyalitki wrote: post #90

flohofwoe wrote: post #75

Library files are not the problem, deploying an SDK as precompiled binary blobs is ;) (I bet that .a/.lib files were originally never really meant for software distribution, but only as intermediate file format between a compiler and linker, both running as part of the same build process)

Yeah, but when the product is an SDK, and customers develop on top of it (using their own toolchains) there isn't a lot left for me to play with.

SDK could ship the source, lol, stop kneecapping your consumers.

Re: The .a file is a relic: Why static archives were a bad idea all along

#97

Post by dmitrygr » Tue, Jul 22, 2025, 7:01 PM UTC

astrobe_ wrote: post #37

TuxSH wrote: post #11

> This design decision at the source level, means that in our linked binary we might not have the logic for the 3DES building block, but we would still have unused decryption functions for AES256. Do people really not know about `-ffunction-sections -fdata-sections` & `-Wl,--gc-sections` (doesn't require LTO)? Why is it used so little when doing statically-linked builds? > Let’s say someone in our library designed th…

Yes, these are really esoteric options, and IIRC GCC's docs say they can be counter-productive.

Esoteric? In embedded, people know of these from BEFORE they stop wearing diapers

Re: The .a file is a relic: Why static archives were a bad idea all along

#98

Post by benreesman » Tue, Jul 22, 2025, 7:12 PM UTC

eyalitki wrote: post #95

benreesman wrote: post #92

Earlier quoted context omitted.

`pkg-config` works great in just about any standard scenario: it puts flags on a compile and link line that have been understood by every C compiler and linker since the 1970s. Here's Bazel consuming it with zero problems, and if you have a nastier problem than a low-latency network system calling `liburing` on specific versions of the kernel built with Bazel? Stop playing. The last thing we need is another failed st…

If someone needs a wrapper for a technology, that modifies the output it provides (like meson and bazel do), maybe there is an issue with said technology. If pkg-config was never meant to be consumed directly, and was always meant to be post processed, then we are missing this post processing tool. Reinventing it in every compilation technology again and again is suboptimal, and at least Make and CMake do not have th…

This was the point of posting the trivial little `pkg.bzl` above: Bazel doesn't need to do all this crazy stuff in `rules_cc` and `rules_foreign_cc`: those are giant piles of internal turf wars within the Blaze team that have spilled onto GitHub.

The reason why we can't have nice things is that nice things are simple and cheap and there's no money or prestige in it. `zig cc` demonstrates the same thine.

That setup:

1. mega-force / sed / patch / violate any build that won't produce compatible / standard archives: https://gist.github.com/b7r6/16f2618e11a6060efcfbb1dbc591e96...

2. build sane pkg-config from CMake vomit: https://gist.github.com/b7r6/267b4401e613de6e1dc479d01e795c7...

3. profit

delivers portable (trivially packages up as `.deb` or anything you want), secure (no heartbleed 0x1e in `libressl`), fast (no GOT games, other performance seppeku) builds. These are zero point zero waste: fully artifact cached at the library level, fully action cached at the source level, fully composable, supporting cross-compilation and any standard compiler.

I do this in real life. It's a few hundred lines of nix and bash. I'm able to do this because I worked on Buck and shit, and I've dealt with Bazel and CMake for years, and so I know that the stuff is broken by design, there is no good reason and no plan beyond selling consulting.

This complexity theatre sells shit. It sure as hell doesn't stop security problems or Docker brainrot or keep cloud bills down.

Re: The .a file is a relic: Why static archives were a bad idea all along

#99

Post by krackers » Tue, Jul 22, 2025, 7:42 PM UTC

dzaima wrote: post #4

How possible would it be to have a utility that merges multiple .o files (or equivalently a .a file) into one .o file, via changing all hidden symbols to local ones (i.e. alike C's "static")? Would solve the private symbols leaking out, and give a single object file that's guaranteed to link as a whole. Or would that break too many assumptions made by other things?

It seems like is precisely what the other commenter implemented? https://news.ycombinator.com/item?id=44645423

Re: The .a file is a relic: Why static archives were a bad idea all along

#100

Post by alextingle » Tue, Jul 22, 2025, 7:58 PM UTC

greenavocado wrote: post #61

TuxSH wrote: post #11

> This design decision at the source level, means that in our linked binary we might not have the logic for the 3DES building block, but we would still have unused decryption functions for AES256. Do people really not know about `-ffunction-sections -fdata-sections` & `-Wl,--gc-sections` (doesn't require LTO)? Why is it used so little when doing statically-linked builds? > Let’s say someone in our library designed th…

> Do people really not know about $OBSCURE_GCC_FLAG? Do you know what you sound like?

If your whole business revolves around shipping static libraries to customers, then surely reading the man page to work out how that can be done is part of the job.

Also, having your library rely on static initialisation doesn't seem like a very sound architectural choice.

Honestly, this just sounds like whining from someone who can't be bothered to read the documentation, and can't get the coders to take him seriously. In the time it took him to write that article, he could have read up on his tools, and probably learned some social skills too.