I've been using vim for roughly half my life at this point. What I've found is that actually using the editor is so thoroughly muscle memory at this point that using another editor, where I actually have to consciously think about how to do something, is incredibly distracting.
It's not like the difference in input speed is really meaningful; as you'll see someone point out in any discussion about this stuff, the bottleneck in programming is thinking.
But what I find will often happen is:
1. I settle on a course of action.
2. I go to actually do it.
3. The editor doesn't do what I expect, because it's not vim.
4. I am momentarily confused, and have to think for a minute about how to do what I want.
5. I lose my train of thought entirely.
6. Rinse, repeat.
So for me it's not so much about any specific thing the editor does to make me productive. I do have opinions about some things, but what's important is that it just fades into the background and lets me think about the problem I'm trying to solve.
Beyond just being "different," I tend to eschew IDEs for much the same reason -- while many of the advanced features seem useful, I find them too distracting to be worth the trouble. Just me and the code, please.