Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Sorry state of dynamic libraries on Linux

macieira.org

Post reply on HN
Print view View original

11–20 of 30 posts

Re: Sorry state of dynamic libraries on Linux

#11

Post by AceJohnny2 »

psykotic wrote: post #6

While we're complaining about the sorry state of Linux, something it's desperately lacking (for those of us who ship binary-only libraries and executables) is an equivalent of Windows's PDBs or OS X's dSYMs for post-mortem debugging without bloating the shipping binaries.

While not as straightforward as on Windows (which isn't developer oriented), it's possible on Linux and standard on most distributions (just look at all the -dbg packages of debian-based distributions). A little googling found these instructions involving objcopy to separate debug symbols: http://www.technovelty.org/code/debug-info.html

Re: Sorry state of dynamic libraries on Linux

#12

Post by [deleted] »

egonk wrote: post #10
psykotic wrote: post #6

While we're complaining about the sorry state of Linux, something it's desperately lacking (for those of us who ship binary-only libraries and executables) is an equivalent of Windows's PDBs or OS X's dSYMs for post-mortem debugging without bloating the shipping binaries.

See http://linux.die.net/man/1/objcopy -> --only-keep-debug Then you can open the core file as usual: gdb executable core (just be sure to have debug files in the same directory)

[deleted]

Re: Sorry state of dynamic libraries on Linux

#13

Post by marshray »

brigade wrote: post #4

I started writing a comment before the site went down (hah, I knew it would be WordPress) and now I'll probably forget to post it there so I'll just post it here. Is the address of externalFunction that is stored in the GOT really resolved to the address of the stub and not the final real address of the function? Because it has to be for data (or code simply wouldn't work), and I don't see why function symbols would…

C and C++ say that a function has the same address in all translation units

Until perhaps very recently, the ISO C and C++ standards didn't actually support dynamic linking. (They didn't officially support multithread concurrency either but flexibility is one of those languages' strengths).

if your final binary is over 2GB the linker could fail to put the symbol within range of the offset and fail

I have had to code around this limitation too but it didn't turn out to be that hard in practice.

How about we optimize for the case where the final binary is 2GB or smaller? :-)

Re: Sorry state of dynamic libraries on Linux

#14

Post by ice799 »

psykotic wrote: post #6

While we're complaining about the sorry state of Linux, something it's desperately lacking (for those of us who ship binary-only libraries and executables) is an equivalent of Windows's PDBs or OS X's dSYMs for post-mortem debugging without bloating the shipping binaries.

hi. linux does have this.

you just strip the debug symbols out (and put them somewhere safe). then write a .gnu_debuglink section to the stripped ELF binary with a CRC that matches the stripped symbols.

once something bad happens: you just take the core dump, the symbols you have tucked away, and you are able to debug just fine.

Re: Sorry state of dynamic libraries on Linux

#15

Post by [deleted] »

brigade wrote: post #4

I started writing a comment before the site went down (hah, I knew it would be WordPress) and now I'll probably forget to post it there so I'll just post it here. Is the address of externalFunction that is stored in the GOT really resolved to the address of the stub and not the final real address of the function? Because it has to be for data (or code simply wouldn't work), and I don't see why function symbols would…

[deleted]

Re: Sorry state of dynamic libraries on Linux

#16

Post by psykotic »

ice799 wrote: post #14
psykotic wrote: post #6

While we're complaining about the sorry state of Linux, something it's desperately lacking (for those of us who ship binary-only libraries and executables) is an equivalent of Windows's PDBs or OS X's dSYMs for post-mortem debugging without bloating the shipping binaries.

hi. linux does have this. you just strip the debug symbols out (and put them somewhere safe). then write a .gnu_debuglink section to the stripped ELF binary with a CRC that matches the stripped symbols. once something bad happens: you just take the core dump, the symbols you have tucked away, and you are able to debug just fine.

Thanks to you and others for jumping in and pointing this out! There's nothing better than being corrected when it means learning something new and solving a long-standing problem.

Re: Sorry state of dynamic libraries on Linux

#17

Post by brigade »

marshray wrote: post #13
brigade wrote: post #4

I started writing a comment before the site went down (hah, I knew it would be WordPress) and now I'll probably forget to post it there so I'll just post it here. Is the address of externalFunction that is stored in the GOT really resolved to the address of the stub and not the final real address of the function? Because it has to be for data (or code simply wouldn't work), and I don't see why function symbols would…

C and C++ say that a function has the same address in all translation units Until perhaps very recently, the ISO C and C++ standards didn't actually support dynamic linking. (They didn't officially support multithread concurrency either but flexibility is one of those languages' strengths). if your final binary is over 2GB the linker could fail to put the symbol within range of the offset and fail I have had to code…

> Until perhaps very recently, the ISO C and C++ standards didn't actually support dynamic linking. (They didn't officially support multithread concurrency either but flexibility is one of those languages' strengths).

How so? They certainly didn't mention it but they shouldn't have to - describing the final linked behaviour is enough and means that whether it was statically or dynamically linked doesn't matter if it produces the same run-time behaviour. Which resulted in a huge mess in the linker for C++.

> How about we optimize for the case where the final binary is 2GB or smaller? :-)

I agree, but compilers should be standards-compliant by default and any such optimizations should be under non-default flags (e.g. -fvisibility-inlines-hidden). But -mcmodel=small is already the default...

Re: Sorry state of dynamic libraries on Linux

#18

Post by marshray »

brigade wrote: post #17
marshray wrote: post #13

Earlier quoted context omitted.

C and C++ say that a function has the same address in all translation units Until perhaps very recently, the ISO C and C++ standards didn't actually support dynamic linking. (They didn't officially support multithread concurrency either but flexibility is one of those languages' strengths). if your final binary is over 2GB the linker could fail to put the symbol within range of the offset and fail I have had to code…

> Until perhaps very recently, the ISO C and C++ standards didn't actually support dynamic linking. (They didn't officially support multithread concurrency either but flexibility is one of those languages' strengths). How so? They certainly didn't mention it but they shouldn't have to - describing the final linked behaviour is enough and means that whether it was statically or dynamically linked doesn't matter if it…

I'd seen references to possible standards issues with dynamic linking, but hadn't really thought about it until you pointed it out: taking the address of a function or data object with linkage is no longer naturally returns the same address in different translation units.

Believe it or not, in C++ a pointer to an object or function is valid as a non-type template parameter. Heck, I bet you can even partially specialize on it.

Re: Sorry state of dynamic libraries on Linux

#19

Post by longlivedeath »

psykotic wrote: post #6

While we're complaining about the sorry state of Linux, something it's desperately lacking (for those of us who ship binary-only libraries and executables) is an equivalent of Windows's PDBs or OS X's dSYMs for post-mortem debugging without bloating the shipping binaries.

Have you looked at Google Breakpad? They have some support for this:

http://code.google.com/p/google-breakpad/wiki/LinuxStarterGu...

Re: Sorry state of dynamic libraries on Linux

#20

Post by gioele »

dkarl wrote: post #5

All this for the possibility of interposition? Yes, it seems so. The impact is there for this little-known and little-used feature. Instead of optimising for the common-case scenario where the symbols are not overridden, the ABI optimises for the corner case. Using LD_PRELOAD isn't so rare and strange that you can propose throwing it out without quantifying its performance cost. I can't recall offhand why I needed it…

> All this for the possibility of interposition? Yes, it seems so. The impact is there for this little-known and little-used feature. Instead of optimising for the common-case scenario where the symbols are not overridden, the ABI optimises for the corner case.

It also looks like PIE+PIC is required if you want a secure system with ASLR: " rel="nofollow">http://blog.flameeyes.eu/2009/11/02/the-pie-is-not-exactly-a....

Flameeyes (Gentoo dev) has sent patches to all the main library developer that make sure only the necessary number of symbols is exposed and as much data as possible is marked as read-only. I think this effort is more valuable than proposing a very unlikely ABI change.

Post reply on HN