If you have a lot of machines, SSH certificates are supported in OpenSSH 5.6+ and are awesome: https://www.digitalocean.com/community/tutorials/how-to-crea... . They allow for centralized management of authentication/authorization without having to touch each machine (beyond an initial trust relationship setup).
It supports OCSP responders as well as CRLs.
It can allow or deny access based on X.509v3 extensions.
Using this and a hardware token with an X.509 certificate you can completely eliminate passwords for most cases and have no ability to leak your keying material (barring HSM vulnerability).
Android and iOS devices have historically had little support for smartcards so that's a downside of you use those devices. There's a smartcard framework in Android but I've not used it since 4.x and it was not useful.
Support on ChromeOS, Linux, Windows, Mac OS X, Solaris, and *BSD has been a lot more reliable.
Also, this is a fairly common usage scenario thanks to HSPD-12 and the tens of millions of affected PKI users.