41–50 of 389 posts
By whom?
Quoting Chomsky:
> These considerations bring up a minor problem with the current LLM enthusiasm: its total absurdity, as in the hypothetical cases where we recognize it at once. But there are much more serious problems than absurdity.
> One is that the LLM systems are designed in such a way that they cannot tell us anything about language, learning, or other aspects of cognition, a matter of principle, irremediable... The reason is elementary: The systems work just as well with impossible languages that infants cannot acquire as with those they acquire quickly and virtually reflexively.
Response from o3:
LLMs do surface real linguistic structure:
• Hidden syntax: Attention heads in GPT-style models line up with dependency trees and phrase boundaries—even though no parser labels were ever provided. Researchers have used these heads to recover grammars for dozens of languages.
• Typology signals: In multilingual models, languages that share word-order or morphology cluster together in embedding space, letting linguists spot family relationships and outliers automatically.
• Limits shown by contrast tests: When you feed them “impossible” languages (e.g., mirror-order or random-agreement versions of English), perplexity explodes and structure heads disappear—evidence that the models do encode natural-language constraints.
• Psycholinguistic fit: The probability spikes LLMs assign to next-words predict human reading-time slow-downs (garden-paths, agreement attraction, etc.) almost as well as classic hand-built models.
These empirical hooks are already informing syntax, acquisition, and typology research—hardly “nothing to say about language.”
Quite a nice overview. For almost any specific measure, you can find something that is better than human at that point. And now LLMs architecture have made possible for computers to produce complete and internally consistent paragraphs of text, by rehashing all the digital data that can be found on the internet. But what we're good as using all of our capabilities to transform the world around us according to an inte…
A calculator is consistent and doesn’t “hallucinate” answers to equations. An LLM puts an untrustworthy filter between the truth and the person. Google was revolutionary because it increased access to information. LLMs only obscure that access, while pretending to be something more.
> It’s as if a biologist were to say: “I have a great new theory of organisms. It lists many that exist and many that can’t possibly exist, and I can tell you nothing about the distinction.” > Again, we’d laugh. Or should. Should we? This reminds me acutely of imaginary numbers. They are a great theory of numbers that can list many numbers that do 'exist' and many that can't possibly 'exist'. And we did laugh when im…
While there's some things in this I find myself nodding along to in this, I can't help but feel it's an a really old take that is super vague and hand-wavy. The truth is that all of the progress on machine learning is absolutely science. We understand extremely well how to make neural networks learn efficiently; it's why the data leads anywhere at all. Backpropagation and gradient descent are extraordinarily powerful. Not to mention all the "just engineering" of making chips crunch incredible amounts of numbers.
Chomsky is extremely ungenerous to the progress and also pretty flippant about what this stuff can do.
I think we should probably stop listening to Chomsky; he hasn't said anything here that he hasn't already say a thousand times for decades.
Earlier quoted context omitted.
Tons of people fall for this too. Are they not reasoning? LLMs can also be bad reasoning machines.
I dont have much use for a bad reasoning machine.
An attempt: They are bad reasoning machines that already are useful in a few domains and they're improving faster than evolutionary speeds. So even if they're not useful today in a domain relevant to you there's a significant possibility they might be in a few months. AlphaEvolve would have been scifi a decade ago.
"It's like if a squirrel started playing chess and instead of "holy shit this squirrel can play chess!" most people responded with "But his elo rating sucks""
> The world’s preeminent linguist Noam Chomsky, and one of the most esteemed public intellectuals of all time, whose intellectual stature has been compared to that of Galileo, Newton, and Descartes, tackles these nagging questions in the interview that follows. By whom?
What is elegant as a model is not always what works, and working towards a clean model to explain everything from a model that works is fraught, hard work.
I don’t think anyone alive will realize true “AGI”, but it won’t matter. You don’t need it, the same way particle physics doesn’t need elegance