Earlier quoted context omitted.
I actually don't understand the last part of the OP story - can someone explain it?
The sending mail server was configured with a zero timeout for connections. If it didn't IMMEDIATELY get a response from the destination mail server it would fail. This immediate failure took 3 milliseconds, a long enough time that some servers could actually respond back before that happened... but if the server was too far away (more than 500 miles) the connection would fail before the first packet could even get t…
Thanks for writing the TL;DR version!