321–330 of 389 posts
Maybe I am missing context, but it seems like he’s defending himself from the claim that we shouldn’t bother studying language acquisition and comprehension in humans because of LLM’s? Who would make such a claim? LLM’s are of course incredible, but it seems obvious that their mechanism is quite different than the human brain. I think the best you can say is that one could motivate lines of inquiry in human understan…
Earlier quoted context omitted.
You know how many old sci-fi settings pictured aliens as bipedal furry animals or lizards? Even to go from that to realistically-intelligent swarms of insects is already difficult. (Of course, there’s plenty of sci-fi where conscious entities manifest themselves as abstract balls of pure energy or the like; except for some reason those balls still think in the same way we do, get assigned the same motivations, someti…
In Star Trek the whole humanoids everywhere thing is an obvious practicality in producing episodes, though. They spent the whole budget on the salt vampire and never recovered.
[0] Genuinely not unlike how a congregation of gelled-together humans is an entity that can achieve much more than an individual human.
Earlier quoted context omitted.
> AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution I would push back on this a little bit. While it has not helped us to understand our own intelligence, it has made me question whether such a thing even exists. Perhaps there are no simple and beautiful natural laws, like those that exists in Physics, that can explain how humans think and…
> Perhaps there are no simple and beautiful natural laws, like those that exists in Physics, that can explain how humans think and make decisions. Isn't Physics trying to describe the natural world? I'm guessing you are taking two positions here that are causing me confusion with your statement: 1) that our minds can be explained strictly through physical processes, and 2) our minds, including our intelligence, are o…
OTOH, consider LLMs as a roomful of monkeys that can communicate to each other, look at words,sentences and paragraphs on posters around the room with a human in the room that gives them a banana when they type out a new word, sentence or paragraph.
You may eventually get a roomful of monkeys that can respond to a new sentence you give them with what seems an intelligent reply. And since language is the creation of humans, it represents an abstraction of the world made by humans.
Earlier quoted context omitted.
The great breakthrough in AI turned out to be LLMs. Nature , for an LLM, is its design: graph, starting weights, etc. Environment , for an LLM, is what happens during training. LLMs are capable of learning grammar entirely from their environment, which suggests that infants are too, which is bad for Chomsky's position that the basics of grammar are baked into human DNA.
LLMs require vastly more data than humans and still struggle with some more esoteric grammatical rules like parasitic gaps. The fact grammar can be approximated given trillions of words doesn't explain how babies learn language from a much more modest dataset.
However, we now have a proof-of-concept that a computer can learn grammar in a sophisticated way, from the ground up.
We have yet to code something procedural that approaches the same calibre via a hard-coded universal grammar.
That may not obliterate Chomksy's position, but it looks bad.
Earlier quoted context omitted.
This is where I'm stuck. For other commentators, as I understand it, Chomsky's talking about well-defined grammar and language and production systems. Think Hofstadter's Godel Escher Bach. Not "folk" understanding of language. I have no understanding or intuition, or even a finger nail grasp, for how an LLM generates, seemingly emulating, "sentences", as though created with a generative grammar. Is any one comparing…
I don't really understand your question but if a deep neural network predicts the weather we don't have any problem accepting that the deep neural network is not an explanatory model of the weather (the weather is not a neural net). The same is true of predicting language tokens.
LLMs (like GPT) and grammars (like Backus–Naur Form) are two different kinds of generative (production) systems, right?
You've been (heroically) explaining Chomsky's criticism of LLMs to other noobs: grammars (theoretically) explain how humans do language, which is very different from how ChatGPT (stochastic parrots) do language. Right?
Since GPT mimics human language so convincingly, I've been wondering if there's any overlap of these two generative systems.
Especially once the (tokenized) training data for GPTs is word based instead of just snippets of characters.
Because I notice grammars everywhere and GPT is still magic to me. Maybe I'd benefit if I could understand GPTs in terms of grammars.
Earlier quoted context omitted.
LLMs require vastly more data than humans and still struggle with some more esoteric grammatical rules like parasitic gaps. The fact grammar can be approximated given trillions of words doesn't explain how babies learn language from a much more modest dataset.
It's not that the invention of LLMs conclusively disproves Chomksy's position. However, we now have a proof-of-concept that a computer can learn grammar in a sophisticated way, from the ground up. We have yet to code something procedural that approaches the same calibre via a hard-coded universal grammar. That may not obliterate Chomksy's position, but it looks bad.
"To characterize a structural analysis of state violence as “apologia” reveals more about prevailing ideological filters than about the critique itself. If one examines the historical record without selective outrage, the pattern is clear—and uncomfortable for all who prefer myths to mechanisms." the fake academic facade, the us diabolism, the unwillingness to see complexity and responsibility in other its all with us forever ..
Earlier quoted context omitted.
It's not that the invention of LLMs conclusively disproves Chomksy's position. However, we now have a proof-of-concept that a computer can learn grammar in a sophisticated way, from the ground up. We have yet to code something procedural that approaches the same calibre via a hard-coded universal grammar. That may not obliterate Chomksy's position, but it looks bad.
That's not the goal of generative linguistics though; it's not an engineering project.
Again, that LLMs can learn to compose sophisticated texts from training alone does not close the case on Chomsky's position.
However, it is a piece of evidence against it. It does suggest, by Occam's razor, that a hardwired universal grammar is the lesser theory.