Unix originated on the PDP-11, a machine with very limited memory and disk space. At that time, this was not only the right solution, it was probably the only solution.
Calling it “a bad idea all along” is undeserved.
The .a file is a relic: Why static archives were a bad idea all along
121–130 of 136 posts
Re: The .a file is a relic: Why static archives were a bad idea all along
#122Earlier quoted context omitted.
PIE code is different than PIC. PIE can assume no interposition.
Sorry what do you mean by "symbol interpolation" and "interposition" in this context? Natively I would assume you can just take the sections out of the shared object and slap them into the executable. They're both position independent so what's the issue? If PIE allows greater assumptions to be made by the compiler/linker than PIC that sounds great for performance, but doesn't imply PIC code won't work in a PIE conte…
PIC code would work where PIE does, but likely perform worse. In shared libraries, calls to any non-static function can't be assumed to be the same function at runtime, since another library linked or using LD_PRELOAD may also define the symbol. Thus all calls to non-static functions must go through the shared library lookup machinery. This prevents inlining opportunites as well. Functions in an executable can't be overwridden in this manner, and override the symbols in shared libraries. Thus PIE code can have cheaper function calls and freely inline functions.
Its not that you couldn't use the PIC code, but it would be better to just recompile with PIE.
Re: The .a file is a relic: Why static archives were a bad idea all along
#123It is unclear to me what the author's point is. Its seems to center on the example of DPDK being difficult to link (and it is a bear, I've done it recently). But its full of strawmen and falsehoods, the most notable being the claims about the deficienies of pkg-config. pkg-config works great, it is just very rarely produced correctly by CMake. I have tooling and a growing set of libraries that I'll probably open sour…
> Static libraries have lots of game-changing advantages, but performance, security, and portability are the biggest ones. No idea how you come to that conclusion, as they are definitively no more secure than shared libraries. Rather the opposite is true, given that you (as end user) are usually able to replace a shared library with a newer version, in order to fix security issues. Better portability is also question…
A hacker can easily replace your shared library with their own malicious version or intercept calls into one as needed. As the number of distinct binary blobs for an application increases, the surface area for attack vectors increases making security a nightmare. Every piece also needs to be individually signed and authenticated adding more complexity to the application deployment.
As the gp mentioned, Static libraries have a lot of advantages by having only one binary to sign, authenticate and lockdown/test/prove the public interface. The idea is extended into the "Unikernel" approach where even the OS becomes part of the single binary which is then deployed to bare-metal (embedded systems) or a Hypervisor.
Re: The .a file is a relic: Why static archives were a bad idea all along
#124Oh, static linking can be lots of "fun". I ran into this interesting issue once. 1. We have libshared. It's got logging and other general stuff. libshared has static "Foo foo;" somewhere. 2. We link libshared into libfoo and libbar. 3. libfoo and libbar then go into application. If you do this statically, what happens is that the Foo constructor gets invoked twice, once from libfoo and once from libbar. And also gets…
But this is expected behaviour. The Linker cannot know about your intent but is "dumb" in that it only follows some simple rules. Both libfoo and libbar have their own copy of the .o from libshared containing the "Foo foo" instance. Thus the .init/.fini sections in libfoo and libbar make calls to the ctor/dtor of their own "Foo foo" instances resulting in the observed two calls in the app.
The way people generally solve this problem is by using a helper class in the library header file which does reference counting for proper initialization/destruction of a single global instance. For an example see std::ios_base::Init in the standard C++ library - https://en.cppreference.com/w/cpp/io/ios_base/Init
To understand the basics of how linking (both static and dynamic) works see;
1) Hongjiu Lu's ELF: From the Programmer's Perspective - https://ftp.math.utah.edu/u/ma/hohn/linux/misc/elf/elf.html
2) Ian Lance Taylor's 20-part linker essay on his blog; ToC here - https://lwn.net/Articles/276782/
Re: The .a file is a relic: Why static archives were a bad idea all along
#125Earlier quoted context omitted.
I think people today often do not understand any more that C like many other things in the UNIX world is a tool, not a complete framework. But somehow people expect a complete convenient framework with batteries included. They see it has a deficiency that C by itself does not provide many things. I see it as one of its major strengths and this is one of the reasons why I prefer it.
Sane building system is pretty much basic thing in modern language You'd expect something as mature as C to have such a thing by default
Thanks for confirming my point ;-)
Re: The .a file is a relic: Why static archives were a bad idea all along
#126> This design decision at the source level, means that in our linked binary we might not have the logic for the 3DES building block, but we would still have unused decryption functions for AES256. Do people really not know about `-ffunction-sections -fdata-sections` & `-Wl,--gc-sections` (doesn't require LTO)? Why is it used so little when doing statically-linked builds? > Let’s say someone in our library designed th…
There's also the other 'old-school' method to compile each function into its own object file, I guess that's why MUSL has each function in its own source file: https://github.com/kraj/musl/tree/kraj/master/src/stdio ...but these days -flto is simply the better option to get rid of unused code and data - and enable more optimizations on top. LTO is also exactly why static linking is strictly better than dynamic linkin…
Or plugins, or hotcode reload techniques, unless people want to go the OS IPC route that seems forgotten to many, while being safer, but naturally more resource demanding.
Re: The .a file is a relic: Why static archives were a bad idea all along
#127Earlier quoted context omitted.
It makes more sense for c++ due to templates, but the header only C library trend is indeed very strange. It's not surprising that people are coming up now who are writing articles about being confused by static linking behavior.
It's not strange at all. You only have one file to keep track of and it does everything, you put the functions in any compilation unit you want, C compilation is basically instant, and putting a bunch of single file libraries into one compilation unit simplifies things further.
That might be why back in 1999 - 2002, I was waiting around 1h for each OS build variant of our product, a mix of Tcl and C native libraries, super fast.
It is only basically instant in toy examples, or optimizations completely disabled.
Re: The .a file is a relic: Why static archives were a bad idea all along
#128Do people who write this kind of pieces with such peremptory titles really believe that they finally came about to understand everything better after decades of ignorance? Chesterton’s Fence yada yada?
Linking works the way it does because of inertia. There's a pretty broad space of possible ways to link (both static and dynamic). Someone once wrote one of them, and it was good enough, so it stuck, and spread, and now everything assumes it. It's far from the only possible way, but you'd have to write new tooling to do it a different way, and that's rarely worth it. The way they selected is pretty reasonable, but no…
Linking is not done the same way in all platforms or compiled programming languages, if anything most of the inercia has been on FOSS OSes.
Re: The .a file is a relic: Why static archives were a bad idea all along
#129Earlier quoted context omitted.
Linking works the way it does because of inertia. There's a pretty broad space of possible ways to link (both static and dynamic). Someone once wrote one of them, and it was good enough, so it stuck, and spread, and now everything assumes it. It's far from the only possible way, but you'd have to write new tooling to do it a different way, and that's rarely worth it. The way they selected is pretty reasonable, but no…
Linking is not done the same way in all platforms or compiled programming languages, if anything most of the inercia has been on FOSS OSes.
Really, on which platform is a static library not a bundle of object files?
Re: The .a file is a relic: Why static archives were a bad idea all along
#130Earlier quoted context omitted.
Linking is not done the same way in all platforms or compiled programming languages, if anything most of the inercia has been on FOSS OSes.
Really, on which platform is a static library not a bundle of object files?
Moving goal posts, you point apparently was about linking as general OS process to make libraries, and not static libraries in particular.
So many differences, not every programming language relies on the OS linker UNIX style, many have had the linking process as part of their own toolchain, outside BSDs/Linux, other vendors have had improvements on their object file formats, linking algorithms, implementations of linking processes, delayed linking on demand.
Bytecode based OSes like Burroughs/ClearPath MCP, OS/400 (IBM i), also have their own approach how linking takes place and so forth.
Some OSes like Windows Phone 8.x used a mixed executable format with bytecode and machine symbols, with final linking step during installation on device.
Oberon used slim binaries with compressed ASTs, JIT compiled on load, or plain binaries. The linker was yet another compiler pass.
Many other examples to look into, across systems research.
Far from "Linking works the way it does because of inertia.".