Chomsky on what ChatGPT is good for (2023)

Restricted to linguistics, LLM's supposed lack of understanding should be a non-sequitur. If the question is whether LLMs have formed a coherent ability to parse human languages, the answer is obviously yes. In fact not just human languages, as seen with multimodality the same transformer architecture seems to work well to model and generate anything with inherent structure.

I'm surprised that he doesn't mention "universal grammar" once in that essay. Maybe it so happens that humans do have some innate "universal grammar" wired in by instinct but it's clearly not _necessary_ to be able to parse things. You don't need to set up some explicit language rules or generative structure, enough data and the model learns to produce it. I wonder if anyone has gone back and tried to see if you can extract out some explicit generative rules from the learned representation though.

Since the "universal grammar" hypothesis isn't really falsifiable, at best you can hope for some generalized equivalent that's isomorphic to the platonic representation hypothesis and claim that all human language is aligned in some given latent representation, and that our brains have been optimized to be able to work in this subspace. That's at least a testable assumption, by trying to reverse engineer the geometry of the space LLMs have learned.

> Where is the research on impossible language that infants can't acquire? A good popsci article would give me leads here.

It's not infants, it's adults but Moro "Secrets of Words" is a book that describes the experiments and is aimed at lay people.

> Even assuming Chomsky's claim is true, all it shows is that LLMs aren't an exact match for human language learning. But even an inexact model can still be a useful research tool.

If it is it needs to be shown, not assumed. Just as you wouldn't by default assume that GPS navigation tells you about insect navigation (though it might somehow).

> Again, a good popsci article would actually support these claims instead of simply asserting them and implying that anyone who disagrees is a simpleton.

He justifies the statement in the previous sentence (which you don't quote) where he says that it is self-evident by virtue of the fact that something exists at the beginning (i.e. it's not empty space). That's the "merest truism". No popsci article is going to help understand that if you don't already.

> is not an explanatory model of the weather (the weather is not a neural net)

I don't follow. Aren't those entirely separate things? The most accurate models of anything necessarily account for the underlying mechanisms. Perhaps I don't understand what you mean by "explanatory"?

Specifically in the case of deep neural networks, we would generally suppose that it had learned to model the underlying reality. In effect it is learning the rules of a sufficiently accurate simulation.

> The most accurate models of anything necessarily account for the underlying mechanisms

But they don't necessarily convey understanding to humans. Prediction is not explanation.

There is a difference between Einstein's General Theory of Relativity and a deep neural network that predicts gravity. The latter is virtually useless for understanding gravity (that's even if makes better predictions).

> Specifically in the case of deep neural networks, we would generally suppose that it had learned to model the underlying reality. In effect it is learning the rules of a sufficiently accurate simulation.

No, they just fit surface statistics, not underlying reality. Many physics phenomena were predicted using theories before they were observed, they would not be in the training data even though they were part of the underlying reality.

> No, they just fit surface statistics, not underlying reality.

I would dispute this claim. I would argue that as models become more accurate they necessarily more closely resemble the underlying phenomena which they seek to model. In other words, I would claim that as a model more closely matches those "surface statistics" it necessarily more closely resembles the underlying mechanisms that gave rise to them. I will admit that's just my intuition though - I don't have any means of rigorously proving such a claim.

I have yet to see an example where a more accurate model was conceptually simpler than the simplest known model at some lower level of accuracy. From an information theoretic angle I think it's similar to compression (something that ML also happens to be almost unbelievably good at). Related to this, I've seen it argued somewhere (I don't immediately recall where though) that learning (in both the ML and human sense) amounts to constructing a world model via compression and that rings true to me.

> Many physics phenomena were predicted using theories before they were observed

Sure, but what leads to those theories? They are invariably the result of attempting to more accurately model the things which we can observe. During the process of refining our existing models we predict new things that we've never seen and those predictions are then used to test the validity of the newly proposed models.