Bayes's Theorem: What's the Big Deal?

I don't like the introduction, mostly because I believe what he ridicules to a degree (AI is an existential risk, I have enjoyed HPMOR more than Rowling's work, and I'm not quite sure that making lots of money and give it away is worse than working directly for whatever cause we want to support).

Bayesian grace

Bayes T-shirt means certain asshole. Well at least it made me smile.

I agree that Bayes' theorem is not that notable. The product rule, more fundamental, is more representative of the laws of probability (the actual basis for bayesianism).

Of course, the "formula for the perfect brain" is computationally intractable…

The association between Bayesianism and Neo-Liberalism looks like an ad-hominem attack —doesn't apply to me, at least.

I can't comment on this "Bayesian revolution". But I'm already suspicious of the whole essay at this point, and cannot trust this paragraph.

Amazing Bayes

What the author fails to acknowledge here is that to the extent it can apply, Probability theory is that amazing. There are, like proofs of it working very broadly, and it relies on very few axioms. (Jayne's work in Probability Theory: the Logic of Science leaves little doubt about that.)

Absence of evidence is evidence of absence. Often very weak evidence, but evidence nonetheless. Whoever believe otherwise doesn't understand probability theory. (Maybe the author conflated "evidence" and "proof", or "evidence" and "strong evidence"?)

The correct application of probability theory is computationally intractable in many cases. I can see how it would be unworkable for historians, who have to juggle with many many kinds of evidence. Not having read Richard Carrier however, I'm not sure this objection is not yet another strawman.

Accusing Bayesianism to be responsible for confirmation bias is ridiculous however. Confirmation bias does not follow probability theory. It often follows an incorrect application of it however.

Less Wrong

Okay, I'm out. The insults are too blunt, came too soon, without any justification so far. SIAI (now MIRI) as a doomsday cult is a strawman. As for LessWrong, Eliezer specifically warned about the dangers of using rationalist's tools for rhetoric purposes —how your own biases can increase when you know about biases.

I'm not taking the effort required to search for and debunk any justification that might come later. He just used up my patience.

Contrast this with an alternative conception of rationality as espoused by David Deutsch.

David Deutsch in his superb books, 'The Fabric Of Theory' and 'The Beginning Of Infinity', argued for a different theory of reasoning than Bayesianism. Deutsch (correctly in my view) pointed out that real science is not based on probabilistic predictions, but on explanations. So real science is better thought of as the growth or integration of knowledge, rather than probability calculations.

So what's wrong with Bayesianism?

Probability theory was designed for reasoning about external observations - sensory data. (for example, "a coin has a 50% chance of coming up heads"). In terms of predicting things in the external world, it works very well.

Where it breaks down is when you try to apply it to reasoning about your own internal thought processes. It was never intended to do this. As statistician Andrew Gelman correctly points out, it is simply invalid to try to assign probabilities to mathematical statements or theories, for instance.

Can an alternative mathematical framework be developed, one more in keeping with the ideas of David Deutsch and the coherence theory of knowledge?

I believe the answer is yes, and I am going to sketch the basic ideas for such a framework.

The basic idea is to separate out levels of abstraction when reasoning (or equivalently, levels of recursion). In my proposed framework, there are 3 levels, and each level gets its own measure of 'truth-value'. All reasoning must terminate in a Boolean truth value (True/False) at the base level but the idea is that different forms of reasoning correspond to different levels of abstraction.

1st level: Boolean logic (True/False)

2nd level: Probability value (0-1)

3rd level: Conceptual coherence (categorization measure)

For full reflection, you need three different numbers: a Boolean value (T/F) at the base, a probability value (0-1) at the next level of abstraction, and an entirely new measure called conceptual coherence at the highest of abstraction.

As a rough working definition of conceptual coherence, I would define it thusly;

"The degree to which a concept coheres with (integrates with) the overall world-model."

It should now be clear what's wrong with Bayesianism! It only gets us to the 2nd level abstraction! There is not just uncertainty about our own knowledge of the world (probability), there is another meta-level of uncertainly; uncertainty about our own reasoning processes, or logical uncertainty. Bayesianism can't help us here. Conceptual coherence can. Lets see how:

All statements of the form:

‘outcome x has probability y’

can be converted into statements about conceptual coherence, simply by redefining ‘x’ as a concept in a world-model. Then the correct form of logical expression is:

‘concept x has coherence value y’.

The idea is that probability values are just special cases of coherence (the notion of coherence is more general than the notion of probabilities).

To conclude, conceptual coherence is the degree with which a concept is integrated with the rest of your world-model, and I think it accurately captures in mathematical terms the ideas that Deutsch was trying express, and is a more powerful method of reasoning than Bayesianism.