Mostly agree: the biggest factor in determining "readability" is simply familiarity of the reader with the particular syntax or style of programming. Learning many different languages allowed me to experience the evolution of code from "how do I even read this" to "well, it's clear and obvious what's going on" - without it changing one bit in the meantime. There's nothing you couldn't learn with enough effort, and the differences between learning time needed to get to mastery are in my experience small, save for a few outliers (IME: J, Forth)
At the same time I believe that there are a few objective metrics that seem to correlate with long term maintainability of a codebase. For example, the more contextual, relevant, and correct(!) information it contains, the easier it is to work with the code, especially once the original authors depart (and they will, sooner or later). Capturing that information and putting it in the code - in whatever way, including comments, diagrams (ascii art and graphical), particular tests and doctests, explicit pre and postconditions, other assertions, log calls - lowers the effort needed to maintain said code.
If the additional information threatens to obscure the the view on what's actually happening, you can refactor it the same way you'd refactor code. If you can introduce helper functions to kick details out of the way, you can also add footnotes and links to files with additional docs to get the level of detail manageable in the most often read code, while still providing enough information about that code.
What do you think? I came to this conclusion based on my experience with learning a very diverse set of programming styles and languages, experience with maintaining long-lived projects at work, and the "Programmer's Brain" book. The book has its moments, though for the most part it's just boring, but it did provide me with a few puzzle pieces I needed to make some sense out of the whole thing.