It doesn't seem shocking to me that AAVE is stigmatized (not that it's right to do so). AAVE re-uses Standard American English words and grammar in different ways. I think it was you up above that brought up the habitual be. If you're not familiar with AAVE, you're probably going to misinterpret it. That stands in contrast to many of the other English dialects, where things are generally either clear from context, or so foreign you have no idea what to make of it and have to ask.
That ability to misinterpret creates confusion, which people don't like. "She be working" could either mean that she's been working a lot recently in AAVE, or a strange way of saying that she is currently at work. Appalachian English, for example, doesn't have pitfalls like that. "They was a-going way over yonder" has grammatical differences, but they don't particularly change the meaning. I would expect most people to be able to infer the meaning of that (presuming they know what "yonder" means).
The second, and probably more important, factor is people's exposure. The portrayal of AAVE in media is not good. I couldn't tell you the last time I saw a character in a show or movie that spoke AAVE, was well educated or prestigious, and wasn't some kind of a shaman, voodoo priest or similar. I don't think I've ever seen a doctor or lawyer on TV that spoke AAVE. The characters that speak AAVE are almost universally poor, usually not very smart, and often involved in either crime or some kind of mysticism. It builds a stereotype around people who speak AAVE, especially since I think most of us have very little exposure to it.
Fwiw, there are other English dialects that are similarly stigmatized. Appalachian English is heavily stigmatized, as is the more general southern dialect, though to less of a degree. The Cockney accent/dialect is similarly stigmatized, and it seems like for similar reasons.