> Accidentally getting non-typechecked code in the compiler, or running into problems with the compiler thinking two equal types are distinct?
Yes, this sort of thing. Basically, you have to have this be deterministic, or you end up with very strange possibilities, possible miscompilations, and in the best case, confusing errors. One option is to simply accept that these things can happen. Another is to restrict what you can do at compile time to ensure that they can't.
An extremely simple example is cross compiling. In Rust, usize is dependent on the architecture you're compiling for. A very simple "just compile and run the program, get the answer, and use it" implementation of compile-time execution will produce a usize of the size of the host, not the target. That's a miscompilation. This example, while being simple, is also simple to fix. But it's an example of how it's not as trivial as "use the compiler to compile the program, then run it." Which maybe isn't how you think of this feature, but how I was, back before I knew anything about this topic :)
> I might be overreacting here though
Nah, I think that you're not wrong. It's just that, when you start applying this super rigorously, you end up in weird places. How can you trust any behavior in a language without a specification? How you can you trust a specification if that specification hasn't been formally proven? How you can trust an implementation of a formally proven specification? How you can you trust that the silicon you're running on does the right thing, even with your bug free, formally proven code?
Everyone chooses somewhere along this axis to be comfortable with. And everyone does something, including "I can ignore these problems because in practice they don't happen to me," to deal with the bits outside of what they consciously choose to focus on.