The City And The City (China Mieville). Two nations, occupying the same physical area, and occupants of either nation have been trained/socialized since birth not to see or interact with the other nation. If they do accidentally, for example with car accident that breaches the psychosocial barrier between them, they must 'unsee' it lest they risk an illegal breach themselves. Sounds like the OP was in the United Stat…
Great book, one of my favorites. Really draws attention to the absurdity of borders and legal status. (I’m not saying we should do away with borders. Just that they carry some weird weird consequences.)