Because SSH requires several seconds to initiate a session, even on a local LAN. Does anyone know why this is the case? Its always baffled me.
The -v flag will help identify where the connection is being delayed.
ssh -v [user@]hostname [command]
Try connecting with the option disabled.
ssh -o GSSAPIAuthentication=no [user@]hostname [command]
You can globally disable GSSAPIAuthentication in ssh config:
echo 'GSSAPIAuthentication=no' >> ~/.ssh/config