Delete an inline function, save 794 kB
randomascii.wordpress.com
Delete an inline function, save 794 kB
1–10 of 45 posts
Re: Delete an inline function, save 794 kB
#2[deleted]
Re: Delete an inline function, save 794 kB
#3With all the work on compilers and programming languages, doesn't it seem like linker and loader technology is ripe for a major overhaul?
This kind of "chaos theory" linker crap is ridiculous.
Re: Delete an inline function, save 794 kB
#4With all the work on compilers and programming languages, doesn't it seem like linker and loader technology is ripe for a major overhaul? This kind of "chaos theory" linker crap is ridiculous.
It seems we know the problem and its solution:
> Ideally the linker would notice the ODR violation, but doing so ends up being expensive – you have to look into every object file to see if any of functions are defined in different ways, and the definition of ‘different’ is not obvious.
For this overhaul you mention, we just need to eliminate "undefined behavior" from our C and C++ implementations. Just. Or permit that [time-]expensive operation to run. Or create some new languages that specify everything so that there is no "undefined behavior."
For the first option (removing undefined behavior), every attempt is met with code that relies on a particular compiler's implementation of "undefined" for a particular operation. For the second (run the expensive operation), the option needs to be available in the linker (I have no idea whether it is) and programmers need to have the patience to let the tools run. For the third, see all the new languages upon which much work is done, especially in the LLVM community (Julia, Rust, Swift).
I believe the renaissance you want is underway. Just not in C/C++ land.
Re: Delete an inline function, save 794 kB
#5With all the work on compilers and programming languages, doesn't it seem like linker and loader technology is ripe for a major overhaul? This kind of "chaos theory" linker crap is ridiculous.
With all the work on compilers and programming languages,
doesn't it seem like linker and loader technology is ripe
for a major overhaul?
Not really that just puts _text_ (raw executable machine code, and binary data) at certain locations in memory and updates some other _text_ to ensure branches/functions point to the right location. This isn't rocket science it works well. All in all loaders are very fast.
Linkers?
I would like to think so. It is just a damn complex problem. The real issue here is legacy support. Object files were a _hack_ to make compilers faster, use less resources, and give old school incremental-recompilation.
To make linkers faster, better, etc. one really needs to move away from opaque object files. I like what the LLVM is doing with LTO but this _kind of_ assumes you have all the rs/c/cpp files your compiling so you can see everything in llvm-ir format.
The real issue is build tools.
Modern C/C++ have a high level tool orchestrate compiling/moving object files/linking object files. In some total you need half a dozen different programs to do all these task (cc, cp, rm, make, sed/awk, ln). Each tool has no clue what everything else is doing.
You need to have that context for the linker to do it's job better. But now the linker has to do the job of all those tools :|
Re: Delete an inline function, save 794 kB
#6With all the work on compilers and programming languages, doesn't it seem like linker and loader technology is ripe for a major overhaul? This kind of "chaos theory" linker crap is ridiculous.
It seems we know the problem and its solution: > Ideally the linker would notice the ODR violation, but doing so ends up being expensive – you have to look into every object file to see if any of functions are defined in different ways, and the definition of ‘different’ is not obvious. For this overhaul you mention, we just need to eliminate "undefined behavior" from our C and C++ implementations. Just. Or permit tha…
The problem with generating ODR violation diagnostics is that it requires the linker to compare all the duplicate definitions to each other not on bit-by-bit basis, but somehow discerning whether they are generated from same input source or functionally equivalent, without embedding some hairy metadata in the object file this seems like something that reduces to halting problem.
This is also one of the many problems that simply ceases to exists once you move away from the C/C++ model of "preprocessor as module system".
Re: Delete an inline function, save 794 kB
#7With all the work on compilers and programming languages, doesn't it seem like linker and loader technology is ripe for a major overhaul? This kind of "chaos theory" linker crap is ridiculous.
For loaders definitely yes! On my local Fedora laptop, qemu takes about 60ms to simply run 'qemu-system-x86_64 -version'. Almost all of the time is taken up in glibc's loader, resolving the ~170 shared libraries.
While 60ms may not sound like a lot, I've been trying to get qemu boot times down to the hundreds of milliseconds (for lightweight VMs and sandboxing). It's been quite successful, but 60ms is now a significant chunk of the total boot time.
Edit: Sure I'm aware I could compile my own custom qemu, or statically link, but that's really not the point. I want qemu to boot very fast for everyone, and each of those shared libraries is a feature that someone, somewhere wants.
Re: Delete an inline function, save 794 kB
#8With all the work on compilers and programming languages, doesn't it seem like linker and loader technology is ripe for a major overhaul? This kind of "chaos theory" linker crap is ridiculous.
With all the work on compilers and programming languages, doesn't it seem like linker and loader technology is ripe for a major overhaul? Loaders? Not really that just puts _text_ (raw executable machine code, and binary data) at certain locations in memory and updates some other _text_ to ensure branches/functions point to the right location. This isn't rocket science it works well. All in all loaders are very fast.…
What strikes me as weird is that I know of exactly one compiler system that got the incrementally compiled modules problem (almost) right and at the same time is AOT compiler without huge runtime: Turbo Pascal. The intermediate files contained both compiler readable definitions as well as object code, the linker was able to link only the required parts without resorting to hacks such as having each function in separate source file (see glibc) and the toolchain was able to generate missing intermediate files even without having to explicitly name them on command line. FPC implements the same mechanism by splitting the compiler metadata and object file into separate on-disk files, which seems like good compromise.
The problem with such approach is that the compiler frontends have to know about each other and produce/consume compatible metadata. For C (and Fortran and...) you an get away without sharing any metadata between separate compilation units and when you cannot (constants, structure layouts) CPP is workable and simple solution, this ignores the problem of inline functions defined in headers, but there is not that much C code that does that.
Re: Delete an inline function, save 794 kB
#9With all the work on compilers and programming languages, doesn't it seem like linker and loader technology is ripe for a major overhaul? This kind of "chaos theory" linker crap is ridiculous.
With all the work on compilers and programming languages, doesn't it seem like linker and loader technology is ripe for a major overhaul? Loaders? Not really that just puts _text_ (raw executable machine code, and binary data) at certain locations in memory and updates some other _text_ to ensure branches/functions point to the right location. This isn't rocket science it works well. All in all loaders are very fast.…
> Not really that just puts _text_ (raw executable machine code, and binary data) at certain locations in memory and updates some other _text_ to ensure branches/functions point to the right location. This isn't rocket science it works well. All in all loaders are very fast.
Really, they are not. In fact resolving symbols in ELF shared libraries is inherently complex and slow. Pick any program which is linked to lots of shared libraries and try running 'time program --version'. 'qemu-system-x86_64 -version' takes 60ms on my laptop.
Re: Delete an inline function, save 794 kB
#10Another (obvious) optimization is to make functions that have to be inline smaller. For example, std::string is used ubiquitously in C++ programs; this change shaved a few MBs of .text from a bunch of large services at FB: https://github.com/facebook/folly/commit/be4c6d6b3e21914df8a... .