>I do, however, somewhat suspect that the crucial difficulty of Emacs comes from its messiness, from the huge number of people writing interesting extensions in it, and how they don't always quite play nicely with each other without a bit of glue code.
There's no excuse for Emacs to be as inaccessible as it is though. What is "crucial difficulty?" You write an awful lot of words to excuse why the world's most powerful text editor does not have a high level of polish or usability out of the box. Usability is not at odds with power: it's complementary. The rich extension ecosystem is amazing, and there are tools in it that cannot be had anywhere else (Magit is my git client for everything, even when using IntelliJ and VSCode). But the number of extensions that exist to fix bad and/or obsolete defaults, and the number of extensions that add basic usability improvements, is really high as well.
I'm not advocating for eliminating what makes Emacs Emacs. I'm advocating for improving the out-of-the-box usability story so that the editor isn't putting people off before they get a chance to understand and experience its power. In short: I don't believe its messiness is an inevitable effect of its power. I think it's a product of the attitude that certain things about it should suck to weed out users who don't fully agree with its ideology.
>By comparison Emacs would of course be the Lisp of text editors. One could try and make Lisp as accessible and nailed down as Java, after all its extreme customisability lends itself to that, but at that point you'd just be writing a poor man's Java instead of Lisp.
I say this as an OCaml and Elixir programmer: there's nothing wrong with writing Java. The world runs on Java, not Lisp. Statistically speaking more Emacs users probably use it to write Java than any other programming language. If we're not going to meet the world where it's at, we will fail eventually.