Much of it is down to nature. Minorities usually don't get preferential treatment when dealing with people who just don't relate naturally with someone they don't share natural similarities with, first-off physically.
I think it's really great that we have strong movements against racism and other forms of stigma stereotypically-related but doing away with it totally remains a job of nature...and people just being naturally different. Am I painting a gloomy-picture, I hope not. I haven't come to a good conclusion on what needs to be done to shake the associations we make in our subconscious minds that affect others unfairly.