It seems odd to me that we still require .o files at all. Given the final program needs to be storable in RAM + Virtual Memory it surprises me that we still need the intermediate step of pushing to the file system only to then immediately reopen and merge those files. Does someone have more info on this? Or is the reason just legacy? Or is the reason just some ideological “single responsibility” thing?
At the highest level of performance needs, if you want to parallelize the build across machines, you're going to need some kind of storage abstraction for intermediate files since translation units are the easiest place to split builds at. In some places, where builds are particularly optimized, there are special distributed filesystems just for object files. In this case, it's not necessarily true that the object fi…
> Being backed by disk locally mainly helps for incrementally building so that you can change one file and only recompile intermediate files for translation units that depend on this file.
For the local incremental usecase I’d love to see a more state-full compiler instead. One that could change the bytes of a binary instead. e.g. give all functions some additional “empty padding”. Then any modifications could directly fit into the binary as needed until some defragmentation process which creates the final output binary.