> Damn, are we doing this wrong?
Yes, and I mean "we" as in "this industry".
I just recently talked to someone whose Swift framework(s) were compiling at roughly 16 lines per second. Spurred on by Jonathan Blow's musings on compile times for Jai[1], I started tinkering with tcc a little. It compiled a generated 200KLOC file (lots of small functions) in around 200 ms.
Then there are Smalltalk and Lisp systems that are always up and pretty much only ever compile the current method.
We also used to have true separate compilation, but that appears to be falling out of favor.
Of course none of these are C++ and they also don't optimize as well etc. Yet, how much of the code really needs to be optimized that well? And how much of that code needs to be recompiled that much?
So we know how to solve this in principle, we just aren't putting the pieces together.
[1] https://www.youtube.com/watch?v=14zlJ98gJKA He mentioned that a decent size game should compile either instantly or in a couple of seconds