Indeed, PGP smartcards should work too, as should cards using the open-source ISOApplet.
I also played around with tpm2-pkcs11 last year and it worked nicely, and has support on a lot of devices like the XPS (which works really well on Linux!)
Your idea for using TPM as host verification makes a lot of sense to me - as it stands right now, I'm playing around with this for a side project right now, and a TPM-backed key which is bound to the right PCRs should give you assurance secure boot is enabled, with your own custom keys, and that it loaded your signed Grub, booting your signed kernel.
Got all the secure boot stuff working, now need to figure out what to use the TPM for - options are remote attestation, where server verifies the attestation signature (but this wouldn't be particularly standardised), network level authentication (actually easier - IIRC NetworkManager supports 802.1x authentication using PKCS11, so you can use tpm2-pkcs11 for that), or disk encryption.
Assuming you mean for the remote-end host key, and assuming your server has a TPM available, I reckon that could be quite interesting, but it doesn't look like openssh supports using PKCS11 for HostKey access. Would need to see if the key is used often, or if it's just used to establish connections (since PKCS11 crypto is usually pretty slow, but fine for one-off authentications).
As a closing note for time-travellers from the future etc, worth remembering TPM is far from perfect, and there's quite a few nice attacks if you can "sniff" the serial lines from the motherboard to the TPM itself. And if using the fTPM (firmware TPM), the regular Intel SGX/TXE holes will likely compromise the TPM security properties.