Earlier quoted context omitted.
It's not 'simple', but it's definitely doable with Yubikey: https://developers.yubico.com/PIV/Guides/SSH_with_PIV_and_PK... I've used it on macOS to use a PIV cert on a yubikey to ssh to remote linux servers. I have NOT used it to ssh-agent chain through, jump-box style though.
If you're already bound to Yubikey, though, any advantage to PKCS#11 over OTP? Theoretically it's an open standard, but there seems to be only one real vendor.
There are many advantages of something like PKCS#11 over One-Time-Passwords (TOTP/HOTP):
1. Since you have an X.509v3 certificate, you have an identity not just an authenticator -- so when you are using this certificate to login to systems, you don't also have to supply an identification (though most systems let you optionally supply one, e.g., if you can login with multiple usernames and the same credential).
2. PKIX covers a lot more use cases than OTP, things like Kerberos (PKINIT), TLS (TLS Client Certificates), SSH (PKIXSSH et al), digital email signing (S/MIME), code signing, commit signing, and more
3. The device is basically a second computer, so it can run software policies on it to do various things like if you authenticate incorrectly a few times in a row it can zero itself out, with OTP systems that has to be coordinated by the various systems validating the OTP token.