I cried as a baby the first time I was put in a group with other babies (most likely of the same color as me).
I think the word you're looking for is xenophobia. And while xenophobia certainly isn't any less of a problem, calling it racism makes it sound far more grandiose than it really is.
Racism is a political (and ideological) issue. Feminists define racism in terms of power dynamics. Supremacists use racism to attribute different qualities to different ethnic groups. Historically racism can be seen as the latent effects of slavery and racial segregation.
Xenophobia is just a mental bias that used to give humans an evolutionary advantage but is becoming increasingly counterproductive in a globalised society. It's a flaw in human thinking that leads us to irrational attitudes but that can be overcome with conscious effort and exposing ourselves to that which is foreign to us.
Conflating the two is one of the main reasons race is still such a hot topic in the US. The US is culturally and ethnically diverse, yes, but that diversity is not equally distributed. In rural white majority areas "blacks" and "Jews" and "orientals" are still as foreign as ever and calling these people racist because they are afraid or unsympathetic of something they only hear about in news reports and on TV is not only unfair but unproductive, no matter how reassuring it may feel to us educated cosmopolitans who are so very progressive and distant from that backwater mindset.
That is why Trump got elected. Republicans who were sick of being told they're racist and Democrats who were disillusioned with the choice of candidates. It didn't even take a record-breaking Republican turnout to get Trump elected, it just took enough Democrats being disinterested in supporting their own candidate.
But I digress. I don't even have any skin in the game -- I'm not even in the US.