The thing about port knocking is that if you're on a host where you don't have the ability to port-knock, then you're not able to connect.
This can turn into a footgun: you're away from your usual device, something happens and you desperately need to connect, but now you can't because all the devices in your vicinity don't have the ability to perform $SECURITY_FEATURE_X so that you can connect, and you're screaming at yourself for adding so much security at the expense of convenience.
This could happen as easily as restricting logins to SSH keys only, and not being able to use your SSH key on whatever host you have available at the time, wishing you'd have enabled password authentication.