Actually, I am not at all convinced by Chomsky's idea of universal grammar. I think here the structuralists and post-structuralists have an important edge (post-structuralism is built on structuralism and the basic thesis is that meaning arises from the system of rules, not the individual rules). IMO Chomsky tries to move away from the basic Structuralist thesis (by Sapir and others) and move towards something else.
So for example when you look across language families you find that most of the constraints follow in an almost mathematical way from other choices. For example, once you have a language which is fairly isolated (as opposed to synthetic or polysynthetic) and doesn't use a lot of inflections, then word order becomes the only reasonable way to express relationships between words. If on the other hand, you have a synthetic language with lots of inflections, you might not need to worry about word order at all (Old English poetry for example).
The options available for polysynthetic languages are different still.
But these aren't the only constraints. Consider for example the way Austronesian languages often essentially "unbound" words by redoubling them, so in Indonesian duduk means to sit, but duduk-duduk means to sit about casually, or informally, or repeatedly (unbounding can occur with regard to time, social structures, etc). Similarly kuru means "horse" and kuru-kuru can either mean "horses" (but only if no numbers are specified) or horse-like (i.e. a sawhorse). Similarly ayam means chicken, and ayam-ayam can either mean chickens or it can mean a kind of fish that people decided is somehow chicken-like ("sea-ckicken?")... On the other hand we would never say dua kuru-kuru to mean "two horses" since that would probably mean "two sawhorses" instead. Instead it is "dua kuru" (translating word-for-word without converting structures, this means "two horse," note the singular).
Where structuralism breaks down (and where post-structuralism is important) is an area which was noted even by Sapir, one of the founders of structuralism, and this is the dynamic quality of grammar over time. Sapir's example from 1912 was the slow, gradual death of "whom" as an objective/dative form of "who." He points out that nobody really would say in every day speech "whom did you see today?" but instead "who did you see today?" and that no cadre of English teachers could reverse that trend. The idea that grammar is not static but changing over time, and hence is fluid is something which really lead to the development of post-structuralism as a general thesis in language study.
I don't think you can get from brain structures directly to grammar for the same reason you can't just treat chemistry as applied quantum physics. The added complexity, esp. combined with neuroplasticity, provides infinite possibilities regarding language structures, provided that we have some way of referencing things (breaking this down strictly into words, phrases, and sentences doesn't work for reasons Sapir points out in his surveys of Native American languages-- all languages may somehow break things down into words and sentences, but there is no natural mapping of objects to words or multiple words to phrases or sentences).
Back to non-standard dialects.... I wonder for example, how the African American Vernacular English (AAVE) phrases in Jason Mraz's song "I'm Yours" get misunderstood. Granted he shifts back and forth between AAVE and Contemporary Standard American English (CSAE) seemingly fluidly. But phrases like "before the cool done run out" have relatively specific meanings and carry a lot more meaning in AAVE while they sound just wrong in CSAE.