Earlier quoted context omitted.
I don't think paradigm shifts have to be 'better' in some march-toward-progress sense, they can be lateral or even regressive in that way and still lead to longer-horizon improvements. I think also what's practically applicable changes constantly. Perhaps we're truly at the End of Science, but empirically we've been wrong every other time we've said that. My money is that there's more race to run.
On that note, Terence Tao gave a good interview to Dwarkesh Patel talking about Kepler. He pointed out that the previous geocentric models were actually more accurate than Kepler's at the time, in part because they'd had so much complexity piled on to solve minor errors. Kepler's theory was more elegant, but at the time it wasn't necessarily a better model. I think important paradigm shifts can often look like this -…
The new simpler tool always competes with highly adapted complex tools to get to a region of value generation.
Starting where it’s greater simplicity, despite less complementary adaptations, is of great advantage.
Then slowly accumulates its own version of practical complements that let it excel overall.