> The medical establishment offers a shiny tempting solution. First, a total unwillingness to trust anything, no matter how plausible it sounds, until it’s gone through an endless cycle of studies and meta-analyses. Isn't this just... science? We learned from the ancient philosophers that really smart people can reason powerfully about a whole lot of things. But - if we want to know whether that reasoning holds up, i…
For example in Machine Learning you can trust a model to work well in the context of a data stream by simply using (cross) validation, and so on (in general avoiding overfiting, and assuming your data stream will continue your training data without changing too much).
Mathematics is an example of a non-experimental science as well: in a way, we can perform purely theoretical experiments where our 'tests' are given by internal consistency. The equivalent of experiments in mathematics for ruling out incorrect statements (inconsistent theories) are proofs and (almost tautologically) the concept of consistency. I think there is some risk that even within math this process derails, but it's only when we start lowering too much the standards for what constitutes a proof. And the concept of proving is so solid we can automate it and computer-verify them. As long as we can (and do) verify what we create, in a sense we can't (or are extremely unlikely to) go off rails.
I think relativity is a good example of a theory that was largely conceived from mathematical principles (from a few observed essential properties of reality). I think in the future this kind of approach will be highly significant to physics, math, philosophy and even social sciences.
It is true though that experiments (and just real life) are very useful to 'keep us in check' in a general, less rigid way as well.
Honestly I think those are the fantastic tool (consistency and proof), fundamentally connected to the notion of truth.