Total western white-male stereotype-infused ignorance here but... Perhaps people avoid muslim women because we don't want them to get in to trouble. I'm surprised to see women dressed like that out in public on their own, because I've been led to believe they need male escorts, and assume that one might be around. if I make eye-contact, I might be contributing to an honor-killing, or perhaps some acid-splashed on her…
That's a very, very small subset of hijab wearers, and you're unlikely to see any of them at a Western mall.