Speaking of non-political side of him: was not he wrong about "innate grammar" necessary to understand langage? LLM do not have such circuitry, yet they somehow work well...
There have been many attempts to model and emulate human syntactic acquisition and processing, but the general consensus is that it cannot be done without presupposing some mechanism that enables hierarchical structure. The number of tokens a child needs to learn syntax is the tiniest fraction of the amount of tokens an LLM is trained on. Humans can also lose parts of their language processing capabilities, without l…
LLMs does have that, or at least it’s very likely that we will eventually be able to manipulate LLMs in a modular way (see https://news.ycombinator.com/item?id=40429540). One point remains: humans learn language with much fewer tokens than LLMs need, which suggests presence of a priori knowledge about the world. The LLM metaphor is finetuning, so babies are born with a base model and then finetuned with environment data, but it’s still within LLM scope.