Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

32-bit x86 Position Independent Code – It's That Bad

ewontfix.com

Post reply on HN
Print view View original

31–40 of 104 posts

Re: 32-bit x86 Position Independent Code – It's That Bad

#31

Post by caf »

zvrba wrote: post #24
deweerdt wrote: post #9

Earlier quoted context omitted.

One of the main reasons for dynamic linking has become irrelevant, I believe: the availability of disk and memory space has grown faster than the size of the binary objects.

This is an oft-repeated mantra, but is it really true? I don't have a linux desktop running here, but try an experiment: start up a desktop environment such as KDE, and write "free -m" and see how much is there under "shared" heading. Without shared libraries, some extra memory corresponding to some multiple of that number would be used. To calculate exactly how much memory is saved by shared libraries, you'd need to…

You could do that calculation in userspace with a stock kernel, by summing up the sizes of the non-writable mappings of the various library files in /proc/*/maps, then using mincore() to find out how much of the libraries are resident.

Re: 32-bit x86 Position Independent Code – It's That Bad

#32

Post by heeen »

deweerdt wrote: post #9
bandrami wrote: post #2

The more I play around with musl (the author's C library) the more I'm convinced dynamic linking was not worth the trouble.

One of the main reasons for dynamic linking has become irrelevant, I believe: the availability of disk and memory space has grown faster than the size of the binary objects.

Are you suggesting that every single calculator, file manager, terminal, editor, package manager and music player ship with their own copy of libQt5* (around 78M) or gtk?

Re: 32-bit x86 Position Independent Code – It's That Bad

#33

Post by roghummal »

pilif wrote: post #21
deweerdt wrote: post #9

Earlier quoted context omitted.

One of the main reasons for dynamic linking has become irrelevant, I believe: the availability of disk and memory space has grown faster than the size of the binary objects.

IMHO, static linking, be it using a mechanism alike what npm is doing or be it plain-old static linking of binaries also means "you link it, you own it". For every package, you link statically, you as the parent package owner become responsible for all security flaws of all the packages you link statically. And by "responsible" I mean: Every dependent security announcement of a dependent package also becomes your sec…

At least on Linux, (commercial) people are packaging shared libs with their binary and using LD_PRELOAD wrappers anyway. "Own-it-by-proxy"-except-you're-shipping-it-so-you-kinda-own-it-anyway.

Dynamically linked, dlopen()-or-equivalent plugin designs are an exception. Otherwise, I'd like to see more statically-linked applications.

Re: 32-bit x86 Position Independent Code – It's That Bad

#34

Post by bandrami »

heeen wrote: post #32
deweerdt wrote: post #9

Earlier quoted context omitted.

One of the main reasons for dynamic linking has become irrelevant, I believe: the availability of disk and memory space has grown faster than the size of the binary objects.

Are you suggesting that every single calculator, file manager, terminal, editor, package manager and music player ship with their own copy of libQt5* (around 78M) or gtk?

Static linking doesn't pull the entire library; it pulls the parts of the library it uses.

That said, QT and GTK probably should be broken into smaller libraries in a perfect world.

Re: 32-bit x86 Position Independent Code – It's That Bad

#35

Post by zvrba »

bandrami wrote: post #30
zvrba wrote: post #24

Earlier quoted context omitted.

This is an oft-repeated mantra, but is it really true? I don't have a linux desktop running here, but try an experiment: start up a desktop environment such as KDE, and write "free -m" and see how much is there under "shared" heading. Without shared libraries, some extra memory corresponding to some multiple of that number would be used. To calculate exactly how much memory is saved by shared libraries, you'd need to…

some extra memory corresponding to some multiple of that number would be used Depends. If I have 7 instances of my terminal emulator loaded (say I hadn't discovered tmux yet or something), the loader can share their .rodata and .text segments, and in many cases it does (YMMV; heuristics apply; void where prohibited; etc.). So a lot of things that are in shared libraries right now might still be only loaded into memor…

I agree, sharing will (most probably) still work on segment-level [0]. But this means that if I have two different executables each linking an identical version of, say QT, in its .text segment, those copies will not be shared.

[0] In fact, certainly. Program loading works by mmaping the executable with MAP_SHARED flag into the process's address space, and the VFS takes care to keep all mappings coherent. The simplest way of keeping mappings coherent is to actually share the backing storage between all mappings. It's the foundation of CoW.

Re: 32-bit x86 Position Independent Code – It's That Bad

#36

Post by zvrba »

caf wrote: post #31
zvrba wrote: post #24

Earlier quoted context omitted.

This is an oft-repeated mantra, but is it really true? I don't have a linux desktop running here, but try an experiment: start up a desktop environment such as KDE, and write "free -m" and see how much is there under "shared" heading. Without shared libraries, some extra memory corresponding to some multiple of that number would be used. To calculate exactly how much memory is saved by shared libraries, you'd need to…

You could do that calculation in userspace with a stock kernel, by summing up the sizes of the non-writable mappings of the various library files in /proc/*/maps, then using mincore() to find out how much of the libraries are resident.

mincore is a system call and does not take a pid parameter. This means it has to be executed by each process individually, which would require injecting code into each running process and executing it in some way.

Unless there's a tool which makes this extremely simple (maybe Intel's pin?), I believe that writing a kernel module is simpler. The module's init function tallies up the pages and writes out the result into the kernel log. Then the module exits.

Re: 32-bit x86 Position Independent Code – It's That Bad

#37

Post by raverbashing »

mappu wrote: post #20
gwu78 wrote: post #13

There was a time when shared libraries were a necessity. Not to mention supporting myriad architectures. That was a different era. Conditions have changed. Of course, the designs and implementations have not. They are still in heavy use, 20 years later. What "just works" is never questioned. I am glad to see some people questioning the old assumptions. Unrelated question: Why does Linux pass arguments in registers in…

>Why does Linux pass arguments in registers instead of using the stack? It's a convention. Depending on your architecture, compiler and API, it uses the stack in a lot of cases (cdecl/stdcall on i386). Linux on AMD64 always uses the SYSV ABI which uses registers as much as possible (i'm assuming for performance reasons, since 'fastcall' does the same on i386).

I didn't know that was the name of it, interesting, though it sounds like an Anachronism

http://en.wikipedia.org/wiki/X86_calling_conventions#System_...

Re: 32-bit x86 Position Independent Code – It's That Bad

#38

Post by zurn »

atesti wrote: post #3

I always wondered why Linux can't just use the same thing as windows for DLLs: Instead of PIC, a big tables of places to patch with the final position. It looks really terrible to lose one precious register and have all this PIC overhead. I don't use mono's ahead of time compilation because it also creates PIC. The JIT has one register more available. But I haven't measured it yet

Linux did this before, but moved to PIC around 1995 as part of the a.out->ELF transition.

ELF actually supports (and always did) non-PIC code on x86 Linux in shared libs, ld.so will do fixups to the code on the fly. See http://eli.thegreenplace.net/2011/11/03/position-independent...

Re: 32-bit x86 Position Independent Code – It's That Bad

#39

Post by q3k »

bandrami wrote: post #2

The more I play around with musl (the author's C library) the more I'm convinced dynamic linking was not worth the trouble.

There is however, an important security benefit from PIC: ASLR. It's not a silver bullet, but can stop a range of vulnerabilities from being reliably exploitable usable unless an memory leak is also available.

Even if you so far as control the instruction pointer as an attacker, you might just not know where to jump to in the target address space...

Re: 32-bit x86 Position Independent Code – It's That Bad

#40

Post by caf »

zvrba wrote: post #36
caf wrote: post #31

Earlier quoted context omitted.

You could do that calculation in userspace with a stock kernel, by summing up the sizes of the non-writable mappings of the various library files in /proc/*/maps, then using mincore() to find out how much of the libraries are resident.

mincore is a system call and does not take a pid parameter. This means it has to be executed by each process individually, which would require injecting code into each running process and executing it in some way. Unless there's a tool which makes this extremely simple (maybe Intel's pin?), I believe that writing a kernel module is simpler. The module's init function tallies up the pages and writes out the result int…

You don't need to run mincore() on target pids - you just need to write a tool that opens(O_RDONLY) and mmaps(PROT_READ) each library file, then calls mincore() on each page in the mapping to find out which pages of the library are loaded shared.

The results of mincore() in one process with a shared mapping of a file are enough to tell you how much of that file is loaded shared system-wide.

Post reply on HN