Dllicious – shared object usage analysis on Linux
wiki.alopex.li
Dllicious – shared object usage analysis on Linux
1–10 of 25 posts
Re: Dllicious – shared object usage analysis on Linux
#2The upper bound is interesting but my limited intuition is it's a very upper bound. LTO/LTCG will drive that down a lot. Last time I was looking at Rust binary sizes things like building the stdlib with heavy LTO/opt size had some pretty big influences on final binary size.
Re: Dllicious – shared object usage analysis on Linux
#3Saving disk space isn’t nearly as exciting as saving CPU cache space which is much more precious. That means that something like libc will almost always be in cache from something calling it.
Re: Dllicious – shared object usage analysis on Linux
#4Excellent trolling. My only gripe:
> So huzzah, you now have some real data for your next Internet Argument
You don't need real data for internet arguments. Make something up and sound convincing, it works at least 60% of the time according to studies by a guy whose wiki blog I read once.
Re: Dllicious – shared object usage analysis on Linux
#5On my NixOS computer filtering for the most used:
10093 libcap.so.2
49344 libc.so.6
27555 libdl.so.2
13296 libffi.so.8
16112 libgcc_s.so.1
10086 libgcrypt.so.20
11352 libglib-2.0.so.0
10128 libgpg-error.so.0
13015 liblzma.so.5
22664 libm.so.6
11337 libpcre.so.1
29942 libpthread.so.0
11036 libresolv.so.2
19510 librt.so.1
17287 libz.so.1
11362 libzstd.so.1
Re: Dllicious – shared object usage analysis on Linux
#6Saving disk space isn’t nearly as exciting as saving CPU cache space which is much more precious. That means that something like libc will almost always be in cache from something calling it.
I think dynamic linking is great for saving (DRAM) memory, but for caches I am having trouble imagining a case in which this would make a practical difference.
You would need two distinct, long-lived dynamically-linked executables, both being hot at the same time, and both having libc (or another common library) in the hotpath at the same time.
To elaborate,
- If two hot processes are the same executable, static linking would yield the same results: They will be mmap()ed to the same physical addresses (and I'm assuming physical indexing or equivalent here, otherwise ASLR really negates any dynamic linking advantage wrt caches).
- If the processes are not hot or not long-lived, everything will be dominated (by orders of magnitude) by media access, context switches, and virtual page allocation.
- Sure, we could imagine a non-hot process trashing the cache for a hot one, and doing it less so with dynamic linking. But again, context switch is the bigger concern. A complete cache flush would make little difference here.
- And I'm not even going to touch on the various cache-timing vuln mitigations out there.
That being said, I am a staunch proponent of dynamic linking being the general case, for the memory and storage savings. Security is a common argument too, and although I'm not expert enough to have an opinion on this one, I do love the traditional distro approach to packages.
Re: Dllicious – shared object usage analysis on Linux
#7> DLL’s add a level of complexity to writing and using software, and newer languages like Rust and Go have eschewed them,
Yeah, no. Every sane compiler developer would always prefer DLLs over static linking because of modularity. It makes the compiler much smaller if you can defer the combination of modules to a linker and thus have proper separate compilation.
Unfortunately, separate compilation with a C-like ABI only works well with a relatively limited monomorphic language. All other languages would have to leave quite a few optimizations on the table or make some suboptimal choices for uniform object representation (compare OCaml's ABI with Rust or Haskell).
Consequently, a modern language would face the tremendous (but I think interesting) task to extend the C-ABI in a way that allows these optimizations and separate compilation. Understandably, developers then play down the importance of separate compilation and choose a global approach.
Re: Dllicious – shared object usage analysis on Linux
#8Saving disk space isn’t nearly as exciting as saving CPU cache space which is much more precious. That means that something like libc will almost always be in cache from something calling it.
I think dynamic linking is great for saving (DRAM) memory, but for caches I am having trouble imagining a case in which this would make a practical difference. You would need two distinct , long-lived dynamically-linked executables, both being hot at the same time, and both having libc (or another common library) in the hotpath at the same time. To elaborate, - If two hot processes are the same executable, static lin…
> You would need two distinct, long-lived dynamically-linked executables, both being hot at the same time, and both having libc (or another common library) in the hotpath at the same time.
Like, for example, bash and a process you are executing in parallel in a loop?
Re: Dllicious – shared object usage analysis on Linux
#9> DLL’s add a level of complexity to writing and using software, and newer languages like Rust and Go have eschewed them, Yeah, no. Every sane compiler developer would always prefer DLLs over static linking because of modularity. It makes the compiler much smaller if you can defer the combination of modules to a linker and thus have proper separate compilation. Unfortunately, separate compilation with a C-like ABI on…
Static linking does not preclude separate compilation though. Go and Rust both use separate compilation with static linking by default (at least last I used them... it's been a while). I know Rust at least used to support dynamic linking too -- it just wasn't the default. And C also supports static linking of course.
Re: Dllicious – shared object usage analysis on Linux
#10The size computation is overblown no? A static link would probably remove most of the symbols as they are probably unused?