[dead]
Chomsky on what ChatGPT is good for (2023)
151–160 of 389 posts
Re: Chomsky on what ChatGPT is good for (2023)
#152The fact that we have figured out how to translate language into something a computer can "understand" should thrill linguists. Taking a word (token) and abstracting it's "meaning" as a 1,000-dimension vector seems like something that should revolutionize the field of linguistics. A whole new tool for analyzing and understanding the underlying patterns of all language! And there's a fact here that's very hard to disp…
Restricted to linguistics, LLM's supposed lack of understanding should be a non-sequitur. If the question is whether LLMs have formed a coherent ability to parse human languages, the answer is obviously yes. In fact not just human languages, as seen with multimodality the same transformer architecture seems to work well to model and generate anything with inherent structure. I'm surprised that he doesn't mention "uni…
Can LLMs actually parse human languages? Or can they react to stimuli with a trained behavioral response? Dogs can learn to sit when you say "sit", and learn to roll over when you say "roll over". But the dog doesn't parse human language; it reacts to stimuli with a trained behavioral response.
(I'm not that familiar with LLM/ML, but it seems like trained behavioral response rather than intelligent parsing. I believe this is part of why it hallucinates? It doesn't understand concepts, it just spits out words - perhaps a parrot is a better metaphor?)
Re: Chomsky on what ChatGPT is good for (2023)
#153The fact that we have figured out how to translate language into something a computer can "understand" should thrill linguists. Taking a word (token) and abstracting it's "meaning" as a 1,000-dimension vector seems like something that should revolutionize the field of linguistics. A whole new tool for analyzing and understanding the underlying patterns of all language! And there's a fact here that's very hard to disp…
Restricted to linguistics, LLM's supposed lack of understanding should be a non-sequitur. If the question is whether LLMs have formed a coherent ability to parse human languages, the answer is obviously yes. In fact not just human languages, as seen with multimodality the same transformer architecture seems to work well to model and generate anything with inherent structure. I'm surprised that he doesn't mention "uni…
Alternatively, what Chomsky was thinking about with his universal grammar idea is something implicitly present in both our minds and an LLM i.e. "it's the wiring, stupid".
I'm not sure there's much evidence for this one way or another at this point.
Re: Chomsky on what ChatGPT is good for (2023)
#154The fact that we have figured out how to translate language into something a computer can "understand" should thrill linguists. Taking a word (token) and abstracting it's "meaning" as a 1,000-dimension vector seems like something that should revolutionize the field of linguistics. A whole new tool for analyzing and understanding the underlying patterns of all language! And there's a fact here that's very hard to disp…
> whether or not an LLM is conscious in the same way as a human being
The problem is... that there is a whole amount of "smart" activities humans do without being conscious of it.
- Walking, riding a bike, or typing on a keyboard happen fluidly without conscious planning of each muscle movement.
- You can finish someone sentence or detect if a sentence is grammatically wrong, often without being able to explain the rule.
- When you enter a room, your brain rapidly identifies faces, furniture, and objects without you consciously thinking, “That is a table,” or “That is John.”
Re: Chomsky on what ChatGPT is good for (2023)
#155The fact that we have figured out how to translate language into something a computer can "understand" should thrill linguists. Taking a word (token) and abstracting it's "meaning" as a 1,000-dimension vector seems like something that should revolutionize the field of linguistics. A whole new tool for analyzing and understanding the underlying patterns of all language! And there's a fact here that's very hard to disp…
Unfortunately you've undermined your point by making sweeping claims about something that is the literal hardest known problem in philosophy (consciousness).
I'm not actually comfortable saying that LLMs aren't conscious. I think there's a decent chance they could be in a very alien way.
I realize that this is a very weird and potentially scary claim for people to parse but you must understand how weird and scary consciousness is.
Re: Chomsky on what ChatGPT is good for (2023)
#156The fact that we have figured out how to translate language into something a computer can "understand" should thrill linguists. Taking a word (token) and abstracting it's "meaning" as a 1,000-dimension vector seems like something that should revolutionize the field of linguistics. A whole new tool for analyzing and understanding the underlying patterns of all language! And there's a fact here that's very hard to disp…
Unfortunately you've undermined your point by making sweeping claims about something that is the literal hardest known problem in philosophy (consciousness). I'm not actually comfortable saying that LLMs aren't conscious. I think there's a decent chance they could be in a very alien way. I realize that this is a very weird and potentially scary claim for people to parse but you must understand how weird and scary con…
[deleted]
Re: Chomsky on what ChatGPT is good for (2023)
#157It’s time to stop writing in this elitist jargon. If you’re communicating and few people understands you, then you’re a bad communicator. I read the whole thing and thought: wait, was there a new thought or interesting observation here? What did we actually learn?
I have problems with Noam Chomsky, but certainly none with his ability to communicate. He is a marvel at speaking extemporaneously in a precise and clear way.
Re: Chomsky on what ChatGPT is good for (2023)
#158Earlier quoted context omitted.
Restricted to linguistics, LLM's supposed lack of understanding should be a non-sequitur. If the question is whether LLMs have formed a coherent ability to parse human languages, the answer is obviously yes. In fact not just human languages, as seen with multimodality the same transformer architecture seems to work well to model and generate anything with inherent structure. I'm surprised that he doesn't mention "uni…
Alternatively, what Chomsky was thinking about with his universal grammar idea is something implicitly present in both our minds and an LLM i.e. "it's the wiring, stupid". I'm not sure there's much evidence for this one way or another at this point.
>is something implicitly present in both our minds and an LLM
The LLM doesn't start with any real structure besides the network of ops though. If there is any induced structure, it's learnable from the data. And given enough data the base network is sufficient to learn the "grammar" of not just human language but more complex CFGs and things you wouldn't traditionally consider "languages" as well (e.g. audio, images). In a sort of chicken/egg scenario, the morasses of data gives rise to the structures needed to parse and generate that data.
Re: Chomsky on what ChatGPT is good for (2023)
#159I'm noticing that leftists overwhelmingly toe the same line on AI skepticism, which suggests to me an ideological motivation.
Then you obviously didn't listen to a word Chomsky has said on the subject. I was quite dismissive of him on LLMs until I realized the utter hubris and stupidity of dismissing Chomsky on language. I think it was someone asking if he was familiar with the Wittgenstein Blue and Brown books and of course because he as already an assistant professor at MIT when they came out. I still chuckle at my own intellectual arroga…
Chomsky has been colossally wrong on universal grammar.
https://www.scientificamerican.com/article/evidence-rebuts-c...
But at least he admits that:
Re: Chomsky on what ChatGPT is good for (2023)
#160Earlier quoted context omitted.
Restricted to linguistics, LLM's supposed lack of understanding should be a non-sequitur. If the question is whether LLMs have formed a coherent ability to parse human languages, the answer is obviously yes. In fact not just human languages, as seen with multimodality the same transformer architecture seems to work well to model and generate anything with inherent structure. I'm surprised that he doesn't mention "uni…
Can LLMs actually parse human languages? Or can they react to stimuli with a trained behavioral response? Dogs can learn to sit when you say "sit", and learn to roll over when you say "roll over". But the dog doesn't parse human language; it reacts to stimuli with a trained behavioral response. (I'm not that familiar with LLM/ML, but it seems like trained behavioral response rather than intelligent parsing. I believe…
You can train LLMs on the output very complex CFGs, and it successfully learns the grammar and hierarchy needed to complete any novel prefix. This is a task much more recursive and difficult than human languages, so there's no reason to believe that LLMs aren't able to parse human languages in the formal sense as well.
And of course empirically LLMs do generate valid English sentences. They may not necessarily be _correct_ sentences in a propositional truth-value sense (as seen by so-called "hallucinations), but they are semantically "well-formed" in contrast to Chomsky's famous example of the failure of probabilistic grammar models, "Colorless green ideas sleep furiously."
I'm not a linguist but I don't think linguistics has ever cared about the truth value of a sentence, that's more under the realm of logic.