You absolutely can be phobic (read: afraid) towards certain cultures and religious values. It is not a secret that people from abrahamic countries more often get their - conservative - values from religion than people in the west nowadays do. As a bisexual man who has a lot of connections / friends in the gay community, we are afraid and for good reason. People like me suffer from violence every day in the middle eas…
As the sibling comment points out, the idea that immigrants or refugees are to blame for recent rises in hate crime is more reflective of how skillfully the right plays minority groups off against each other than it is of any facts on the ground. They love it when gays are more worried about 'foreign' Muslim immigrants than they are about the domestic politicians who are actually gearing up to pass anti-LGBT legislation.