> I wish the barriers to using new architectures were lower.
> For instance, suppose binaries were typically distributed in a platform-agnostic format, like LLVM intermediate representation or something equivalent.
We're doing a pretty good job on portability these days already. Well-written Unix applications in C/C++ will compile happily for any old ISA and run just the same. Safe high-level languages like JavaScript, Java, and Safe Rust are pretty much ISA-independent by definition, it's 'just' a matter of getting the compilers and runtimes ported across.
Adopting LLVM IR for portable distribution, probably isn't the way forward. I don't see that it adds much compared to compiling from source, and it's not what it's intended for. (LLVM may wish to change the representation in a subsequent major version, for instance.)
For programs which are architecture-sensitive by nature, such as certain parts of kernels, there are no shortcuts. Or, rather, I'm confident the major OSs already use all the practical shortcuts they can think up.
> When you run your program the first time, it's compiled to native code for your architecture and cached for later use.
Source-based package management systems already give us something a lot like this.
There are operating systems that take this approach, such as Inferno. [0] I like this HN comment on Inferno [1]: kernels are the wrong place for 'grand abstractions' of this sort.
> I realize I've sort of just re-invented Javascript
Don't be too harsh on yourself, JavaScript would be a terrible choice as a universal IR!
> [[ to the bulk of your second paragraph ]]
In the Free and Open Source world, we're already free to recompile the whole universe. The major distros do so as compiler technology improves.
> Processors would be able to compete directly with each other without regard to vendor-lock-in.
For most application-level code, we're already there. For example, your Java code will most likely run just as happily on one of Amazon's AArch64 instances as on an AMD64 machine. In the unlikely case you encounter a bug, well, that's pretty much always a risk, no matter which abstractions we use.
[0] https://en.wikipedia.org/wiki/Inferno_(operating_system)
[1] https://news.ycombinator.com/item?id=9807777