> using key based auth is so much more secure than changing the port
Is it really? Once you get to a sensible password length (say, >14 characters) haven't you effectively made brute force and dictionary attacks impossible anyway?
> password auth = me having to type my password everytime, likely to have a less secure password
If you use a password often enough, I find the time difference between an 8 character and a 16 character password immeasurably small. It's all muscle memory anyway.
>key based auth = me never having to type it in
Perhaps I'm clinging to a false sense of security, but I find the idea of anyone with access to my local machine being able to access my server with 0 additional credentials really disconcerting.
Perhaps in the theoretical world if they can access your machine it's game over anyway because they can install a keylogger. In the real world, I think there's a fairly large population of people who might access your server who wouldn't/couldn't install a keylogger to steal your password.