If you're talking about machine-learnability of languages then there's two frameworks that are relevant: Language Identification in the Limit and PAC-Learning.
Language Identification in the Limit in short tells us that if there is an automaton equivalent to human language then, if it's at most a regular automaton it can be identified ("learned") by a number of positive only examples approaching infinity, and if it's above regular then a number of negative examples approaching infinity is also needed to identify it. Chomsky based his "Poverty of the Stimulus" argument about linguistic nativism (the built-in "language faculty" of humans) on this result, known as Gold's Result after Mark E. Gold who proved it in the setting of Inductive Inference in 1964. Gold's result is not controversial, but Chomsky's use of it has seen no end of criticism, many from the computational linguistics community (including people in it that have been great teachers to me, without having ever met me, like Charniak, Manning and Schutze, and Jurafsky and Martin) [1].
Those critics generally argue that human language can be learned like everything and anything else: with enough data drawn from a distribution assumed identical to the true distribution of the data in the concept to be learned, and allowing a finite amount of error with a given probability, i.e. under Probably Approximately Correct Learning assumptions, the learning setting introduced by Leslie Valiant in 1984, that replaced Inductive Inference and that serves as the theoretical basis of modern statistical machine learning, in the rare cases where someone goes looking for one. Around the same time that Valiant was describing PAC-Learning, Vapnik and Chervonenkis were developing their statistical learning theory behind the Iron Curtain and if you take a machine learning course in school you'll learn about the VC Dimension and wonder what's that got to do with AI and LLMs.
The big question is how relevant is all this to a) human language and b) learning human language with an LLM. Is there an automaton that is equivalent to human language? Is human language PAC-learnable (i.e. from a polynomial number of examples)? There must be some literature on this in the linguistics community, possibly the cognitive science community. I don't see these questions asked or answered in machine learning.
Rather, in machine learning people seem to assume that if we throw enough data and compute at a problem it must eventually go away, just like generals of old believed that if they sacrifice enough men in a desperate assault they will eventually take Elevation No. 4975 [2]. That's of course ignoring all the cases in the past where throwing a lot of data and compute at a problem either failed completely -which we usually don't hear anything about because nobody publishes negative results, ever- or gave decidedly mixed results, or hit diminishing returns; as a big example see DeepMind's championing of Deep Reinforcement Learning as an approach to real world autonomous behaviour, based on the success of the approach in virtual environments. To be clear, that hasn't worked out and DeepMind (and everyone else) has so far failed to follow the glory of AlphaGo and kin with a real-world agent.
So in short, yeah, there's a lot to say that we may never have enough data and compute to achieve a good enough approximation of human linguistic ability with a large language model, or something even larger, bigger, stronger, deeper, etc.
__________________
[1] See: https://languagelog.ldc.upenn.edu/myl/ldc/swung-too-far.pdf for a history of the debate.
[2] https://youtu.be/MWS5MfJUbUg?si=qovJBV1sFDbjJf19