I'm sorry but this is near unusable since the death of XUL and introduction of WebExtensions. I was a truly avid user of VimFX which did the same thing. Vimium these days does not work when a page hasn't finished loading and it doesn't work on blank pages (about:blank) or any other "system page" (like the preferences or addons). The "o" key can no longer highlight the address bar but brings up a non-native address ba…
I've never experienced your problems, Vimium works on any tab, indipendently from the others.
I don't understand your glitches, really.