It only applies to sites outside the EU if the site "envisages" serving EU customers. The suggested test for this is if the site offers foreign language versions of the site for EU countries, allows payments in EU currencies, etc. For sites in the US that are intended for a US audience, compliance is unnecessary, regardless of the fact that some traffic may be coming from the EU.
From [1]:
Recital 23 provides a further clarification for cases where it’s unclear if a firm offers goods and services to EU data subjects:
Whereas the mere accessibility of the controller’s, processor’s or an intermediary’s website in the Union, of an email address or of other contact details, or the use of a language generally used in the third country where the controller is established, is insufficient to ascertain such intention, factors such as the use of a language or a currency generally used in one or more Member States with the possibility of ordering goods and services in that other language, or the mentioning of customers or users who are in the Union, may make it apparent that the controller envisages offering goods or services to data subjects in the Union.
[1] https://www.gdpreu.org/the-regulation/who-must-comply/