Key takeaway: > And as several correspondents have reminded me already -- switching your sshd to keys only authentication will let you sleep better at night. Even with fail2ban and limited retries, there's no excuse for using password-based authentication anymore. Use an SSH key, protect the key with a password, and turn off password login on all your servers. Other than that, the main gist of this post is: on most p…
Then you're screwed when you need to log on to your server from some random machine that doesn't have your key installed.
If you can login from "some random machine" that means your password is terribly weak as well. Use a password manager, generate passwords with over 100 bits of entropy, and then you can't log into anything without it present. Now start carrying around your password manager on an encrypted thumbdrive, along with your gpg key and private ssh key with the decryption password + password manager password being the only ones you remember.
Any password a human creates and remembers is a problem. The best mitigation is to only remember passwords that are only useful in conjunction with physical access (the password manager / decryption ones).