A brief introduction to the beauty of Information Theory

A brief introduction to the beauty of Information Theory
notamonadtutorial.com

One interesting thing I can say is that while subsets of a set are the language of probability and of logic (the existential quantifier and the universal quantifier are the two adjoints of the inverse image function), we have the dual to be the language of entropy: partitions of a set.

You can unify the two via a self-dual set theory, where you essentially replace the notions of subset (i.e., subobjects) and partitions (i.e., quotient objects) with an abstract subobject notion. This was part of the basis for my MSc and then my supervisor and a colleague of mine [1] showed that you can have what they call a Noetherian form over the category of sets. It is essentially by taking the pullback of the subobject and quotient object functors, though I can't remember if it is the usual pullback when seen as a category of categories.

[1] This is not all published, but you could have a look at the papers of Zurab Janelidze. The key point was his work on functorially self-dual group theory.