I tried to switch to EMacs a year or so ago. I used it for a couple months and I enjoyed the configurability in elisp. One of the last straws for me was that when scrolling down, the cursor moved to stay on the screen. This is one of the quirks of it being a TUI originally. (There is of course a package to for this functionality, but I found it would break frequently.)
As I move through the document, I already have the location of the cursor origin saved in a register and can jump there, or any of several other locations with a single keypress.
I've always considered the mouse & menu system to be an emergency backup for beginners who have hit a wall. They will probably never be first class citizens.
What editor do you use?