Math, computation, grammars, are just models, syntax, that we sometimes use to consciously express things about the world. We could assign syntax to a system composed of whatever thing and say that is Turing-complete or that follows the productions A->B and B->BB of G_1, an imaginary grammar, just because we can assign to it certain symbols as inputs and others as outputs. Numbers, symbols, don't exist in the real world. Math entities only exist as syntax, as human artifacts. They're language entities. That's why they're based on axioms.
I don't see the point in saying that syntax is physical - I mean, the neurons firing in our brains because of photons getting in our eyes explain how I recognized the shape of a car, not that a 'hypothetical 2D array' gets 'processed' with some 'gradient-based algorithm' unconsciously in our brain. It doesn't make biological sense. Turing didn't even defined what a symbol or a computation is in terms of Physics, he just made a (very cool and beautiful, by the way) model.
Chomsky is simply wrong. He's just thinking here that language is suddenly physical, that we have a grammar-processing CPU in our brains, a form of a Universal Turing Machine! Can this number, '1', be physically somewhere in the world, taking in account modern physics and not some weird Platonism? or we just use it as language to refer to physical entities to talk about them? I do think the latter is the truth. Grammars are math entities which we sometimes use to refer to certain things written in books or other spoken things that just happen to have some physical shape in form of sound waves (words) or even to refer to an imaginary entity we've created with math, just like abstract machines. I don't think there's some weird computational thing going on in my head, that's just an oxymoron. Just neurons firing, blood being pumped, neurotransmitters being sent, in all its glorious and mysterious nature. The mind, on the other hand, exists as a high-level feature of some parts of the bigger brain system, it's caused by real things; that mind that lets us do conscious calculations. But that 'Universal Grammar' wouldn't be inside my mind anyway because it's supposed to be unconscious. It wouldn't be a feature of the mind-spanning parts of the brain anyway.
In the end, it's just the whole Nietzschean critique of mistaking language with reality all over again.
I cannot recommend John Searle's work on the subject enough, like the book 'The Rediscovery Of The Mind'. He explains all of this way way better than me.
P.S: Yes, if you think hard about it, this logically implies that a Strong AI won't ever exist. Sorry, Skynet!
P.S.2: Sorry for the long post and possible grammatical mistakes, I'm not a native. And I sincerely hope you enjoyed this post even if you don't agree with me. Have a nice day!