Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The .a file is a relic: Why static archives were a bad idea all along

medium.com

Post reply on HN
Print view View original

11–20 of 136 posts

Re: The .a file is a relic: Why static archives were a bad idea all along

#11

Post by TuxSH »

> This design decision at the source level, means that in our linked binary we might not have the logic for the 3DES building block, but we would still have unused decryption functions for AES256.

Do people really not know about `-ffunction-sections -fdata-sections` & `-Wl,--gc-sections` (doesn't require LTO)? Why is it used so little when doing statically-linked builds?

> Let’s say someone in our library designed the following logging module: (...)

Relying on static initialization order, and on runtime static initialization at all, is never a good idea IMHO

Re: The .a file is a relic: Why static archives were a bad idea all along

#12

Post by tux3 »

benreesman wrote: post #10
dzaima wrote: post #4

How possible would it be to have a utility that merges multiple .o files (or equivalently a .a file) into one .o file, via changing all hidden symbols to local ones (i.e. alike C's "static")? Would solve the private symbols leaking out, and give a single object file that's guaranteed to link as a whole. Or would that break too many assumptions made by other things?

I routinely tear apart badly laid-out .a files and re-ar them into something useful. It's a few lines of bash.

This works, but scripting with the ar tool is annoying because it doesn't handle all the edge cases of the .a format.

For instance if two libraries have a source file foo.c with the same name, you can end up with two foo.o, and when you extract they override each other. So you might think to rename them, but actually this nonsense can happen with two foo.o objects in the same archive.

The errors you get when running into these are not fun to debug.

Re: The .a file is a relic: Why static archives were a bad idea all along

#13

Post by stabbles »

tux3 wrote: post #2

I actually wrote a tool a to fix exactly this asymmetry between dynamic libraries (a single object file) and static libraries (actually a bag of loose objects) I never really advertised it, but what it does is take all the objects inside your static library, and tells the linker to make a static library that contains a single merged object. https://github.com/tux3/armerge The huge advantage is that with a single obje…

It sounds interesting, but I think it's better if a linker could resolve dependencies of static libraries like it's done with shared libraries. Then you can update individual files without having to worry about outdated symbols in these merged files.

Re: The .a file is a relic: Why static archives were a bad idea all along

#14

Post by throwawayffffas »

benreesman wrote: post #8

It is unclear to me what the author's point is. Its seems to center on the example of DPDK being difficult to link (and it is a bear, I've done it recently). But its full of strawmen and falsehoods, the most notable being the claims about the deficienies of pkg-config. pkg-config works great, it is just very rarely produced correctly by CMake. I have tooling and a growing set of libraries that I'll probably open sour…

Couldn't agree more with you the whole reason docker exists is to avoid having to deal with dynamic libraries we package the whole userland and ship it just to avoid dealing with different dynamic link libraries across systems.

Re: The .a file is a relic: Why static archives were a bad idea all along

#15

Post by benreesman »

tux3 wrote: post #12
benreesman wrote: post #10

Earlier quoted context omitted.

I routinely tear apart badly laid-out .a files and re-ar them into something useful. It's a few lines of bash.

This works, but scripting with the ar tool is annoying because it doesn't handle all the edge cases of the .a format. For instance if two libraries have a source file foo.c with the same name, you can end up with two foo.o, and when you extract they override each other. So you might think to rename them, but actually this nonsense can happen with two foo.o objects in the same archive . The errors you get when running…

This is the nastiest one in my `libmodern-cpp` suite: https://gist.github.com/b7r6/31a055e890eaaa9e09b260358da897b....

It took a few minutes, probably has a few edge cases I haven't banged out yet, and now I get to `-l` and I can deploy with `rsync` instead of fucking Docker or something.

I take that deal.

Re: The .a file is a relic: Why static archives were a bad idea all along

#16

Post by tux3 »

stabbles wrote: post #13
tux3 wrote: post #2

I actually wrote a tool a to fix exactly this asymmetry between dynamic libraries (a single object file) and static libraries (actually a bag of loose objects) I never really advertised it, but what it does is take all the objects inside your static library, and tells the linker to make a static library that contains a single merged object. https://github.com/tux3/armerge The huge advantage is that with a single obje…

It sounds interesting, but I think it's better if a linker could resolve dependencies of static libraries like it's done with shared libraries. Then you can update individual files without having to worry about outdated symbols in these merged files.

If you mean updating some dependency without recompiling the final binary, that's not possible with static linking.

However the ELF format does support complex symbol resolution, even for static objects. You can have weak and optional symbols, ELF interposition to override a symbol, and so forth.

But I feel like for most libraries it's best to keep it simple, unless you really need the complexity.

Re: The .a file is a relic: Why static archives were a bad idea all along

#17

Post by dzaima »

Joker_vD wrote: post #6
dzaima wrote: post #4

How possible would it be to have a utility that merges multiple .o files (or equivalently a .a file) into one .o file, via changing all hidden symbols to local ones (i.e. alike C's "static")? Would solve the private symbols leaking out, and give a single object file that's guaranteed to link as a whole. Or would that break too many assumptions made by other things?

Like, a linker, with "objcopy --strip-symbols" run as the post-step? I believe you can do this even today.

--localize-hidden seems to be more what I was thinking of. So this works:

    ld --relocatable --whole-archive crappy-regular-static-archive.a -o merged.o
    objcopy --localize-hidden merged.o merged.o
This should (?) then solve most issues in the article, except that including the same library twice still results in an error.

Re: The .a file is a relic: Why static archives were a bad idea all along

#18

Post by benreesman »

throwawayffffas wrote: post #14
benreesman wrote: post #8

It is unclear to me what the author's point is. Its seems to center on the example of DPDK being difficult to link (and it is a bear, I've done it recently). But its full of strawmen and falsehoods, the most notable being the claims about the deficienies of pkg-config. pkg-config works great, it is just very rarely produced correctly by CMake. I have tooling and a growing set of libraries that I'll probably open sour…

Couldn't agree more with you the whole reason docker exists is to avoid having to deal with dynamic libraries we package the whole userland and ship it just to avoid dealing with different dynamic link libraries across systems.

Right, the popularity of Docker is proof of what users want.

The implementation of Docker is proof of how much money you're expected to pay Bezos to run anything in 2025.

Re: The .a file is a relic: Why static archives were a bad idea all along

#19

Post by amluto »

tux3 wrote: post #2

I actually wrote a tool a to fix exactly this asymmetry between dynamic libraries (a single object file) and static libraries (actually a bag of loose objects) I never really advertised it, but what it does is take all the objects inside your static library, and tells the linker to make a static library that contains a single merged object. https://github.com/tux3/armerge The huge advantage is that with a single obje…

Is there any actual functional difference between the author’s proposed ET_STAT and an appropriately prepared ET_RET file?

For that matter, I’ve occasionally wondered if there’s any real reason you can’t statically link an ET_DYN (.so) file other than lack of linker support.

Re: The .a file is a relic: Why static archives were a bad idea all along

#20

Post by EE84M3i »

Something I've never quite understood is why can't you statically link against an so file? What specific information was lost during the linking phase to create the shared object that presents that machine code from being placed into a PIE executable?
Post reply on HN