Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chomsky on what ChatGPT is good for (2023)

chomsky.info

Post reply on HN
Print view View original

211–220 of 389 posts

Re: Chomsky on what ChatGPT is good for (2023)

#211

Post by FilosofumRex »

0xbadcafebee wrote: post #152
krackers wrote: post #142

Earlier quoted context omitted.

Restricted to linguistics, LLM's supposed lack of understanding should be a non-sequitur. If the question is whether LLMs have formed a coherent ability to parse human languages, the answer is obviously yes. In fact not just human languages, as seen with multimodality the same transformer architecture seems to work well to model and generate anything with inherent structure. I'm surprised that he doesn't mention "uni…

Can LLMs actually parse human languages? Or can they react to stimuli with a trained behavioral response? Dogs can learn to sit when you say "sit", and learn to roll over when you say "roll over". But the dog doesn't parse human language; it reacts to stimuli with a trained behavioral response. (I'm not that familiar with LLM/ML, but it seems like trained behavioral response rather than intelligent parsing. I believe…

Language and intelligence are mostly orthogonal to each other and development of linguistic skills appeared very late in human evolutionary terms.

Babies and in particular Deaf babies understand and communicate significant amount of information w/o parsing sentences. Dogs don't parse human speech, they associate an emotion to the particular sound and body language exhibited to them, repeatedly.

Re: Chomsky on what ChatGPT is good for (2023)

#212

Post by papaver-somnamb »

There was an interesting debate where Chomsky took a position on intelligence being rooted in symbolic reasoning and Asimov asserted a statistical foundation (ah, that was not intentional ;).

LLM designs to date are purely statistical models. A pile, a morass of floating point numbers and their weighted relationships, along with the software and hardware that animates them and the user input and output that makes them valuable to us. An index of the data fed into them, different from a Lucene or SQL DB index made from compsci algorithms & data structure primitives. Recognizable to Azimov's definition.

And these LLMs feature no symbolic reasoning whatsoever within their computational substrate. What they do feature is a simple recursive model: Given the input so far, what is the next token? And they are thus enabled after training on huge amounts of input material. No inherent reasoning capabilities, no primordial ability to apply logic, or even infer basic axioms of logic, reasoning, thought. And therefore unrecognizable to Chomsky's definition.

So our LLMs are a mere parlor trick. A one-trick pony. But the trick they do is oh-so vastly complicated, and very appealing to us, of practical application and real value. It harkens back to the question: What is the nature of intelligence? And how to define it?

And I say this while thinking of the marked contrast of apparent intelligence between an LLM and say a 2-year age child.

Re: Chomsky on what ChatGPT is good for (2023)

#213

Post by Workaccount2 »

hyperadvanced wrote: post #186
CamperBob2 wrote: post #173

Earlier quoted context omitted.

They do. Spend some time using a modern reasoning model. There is a class of interesting problems, nestled between trivial ones whose answers can simply be regurgitated and difficult ones that either yield nonsense or involve tool use, that transformer networks can absolutely, incontrovertibly reason about.

Reason about: sure. Independently solve novel ones without extreme amounts of guidance: I have yet to see it. Granted, for most language and programming tasks, you don’t need the latter, only the former.

99.9% of humans will never solve a novel problem. It's a bad benchmark to use here

Re: Chomsky on what ChatGPT is good for (2023)

#214

Post by Calavar »

irrational wrote: post #204

I have a degree in linguistics. We were taught Chomsky’s theories of linguistics, but also taught that they were not true. (I don’t want to say what university it was since this was 25 years ago and for all I know that linguistics department no longer teaches against Chomsky). The end result is I don’t take anything Chomsky says seriously. So, it is difficult for me to engage with Chomsky’s ideas.

I don't have a degree in linguistics, but I took a few classes about 15 years ago, and Chomsky's works were basically treated as gospel. Although my university's linguistics faculty included several of his former graduate students, so maybe there's a bias factor. In any case, it reminds me of an SMBC comic about how math and science advance over time [1]

[1] https://smbc-wiki.com/index.php/How-math-works

Re: Chomsky on what ChatGPT is good for (2023)

#215

Post by xwolfi »

AIorNot wrote: post #187
xwolfi wrote: post #177

Earlier quoted context omitted.

An LLM will get ... what exactly ? The ability to reorder its sentences ? The LLM doesn't think, doesn't understand, doesn't know what matters more than not, doesn't use what it learns, doesn't expand what it learns to new knowledge, doesn't enjoy reading that book and doesn't suffer through it. So what is it really gonna do with a book, that LLM ? Reorder its internal matrix to be a little bit more precise when auto…

LLM models are to a large extent neuronal analogs of human neural architecture - of course they reason The claim of the “stochastic parrot” needs to go away Eg see: https://www.anthropic.com/news/golden-gate-claude I think the rub is that people think you need consciousness to do reasoning, I’m NOT claiming LLMs have consciousness or awareness

They are really not neuronal analogs, reasoning is far from what they do. If they reasoned, they'd stick to their guns more readily, but try to contradict an LLM and it will make any logic leap you ask it too.

If you debate with me, I'll keep reasoning on the same premises and usually the difference between two humans is not in reasoning but in choice of premises.

For instance you really want here to assert that LLM are close to human, I want to assert they're not - truth is probably in between but we chose two camps. We'll then reason from these premises, reach antagonistic conclusions and slowly try to attack each other point.

An LLM cannot do that, it cannot attack your point very well, it doesn't know how to say you're wrong, because it doesn't care anyway. It just completes your sentences, so if you say "now you're wrong, change your mind" it will, which sounds far from reasoning to me, and quite unreasonable in fact.

Re: Chomsky on what ChatGPT is good for (2023)

#216

Post by wat10000 »

hackinthebochs wrote: post #174
0xbadcafebee wrote: post #152

Earlier quoted context omitted.

Can LLMs actually parse human languages? Or can they react to stimuli with a trained behavioral response? Dogs can learn to sit when you say "sit", and learn to roll over when you say "roll over". But the dog doesn't parse human language; it reacts to stimuli with a trained behavioral response. (I'm not that familiar with LLM/ML, but it seems like trained behavioral response rather than intelligent parsing. I believe…

LLMs are modelling the world, not just "predicting the next token". They are certainly not akin to parrots. Some examples here[1][2][3]. Anyone claiming otherwise at this point is not arguing in good faith. [1] https://arxiv.org/abs/2405.15943 [2] https://x.com/OwainEvans_UK/status/1894436637054214509 [3] https://www.anthropic.com/research/tracing-thoughts-language...

Maybe it takes some world modeling to do it as well as they do, but ultimately they are just predicting the next token. These things are not mutually exclusive.

Re: Chomsky on what ChatGPT is good for (2023)

#217

Post by largbae »

windexh8er wrote: post #207
irrational wrote: post #204

I have a degree in linguistics. We were taught Chomsky’s theories of linguistics, but also taught that they were not true. (I don’t want to say what university it was since this was 25 years ago and for all I know that linguistics department no longer teaches against Chomsky). The end result is I don’t take anything Chomsky says seriously. So, it is difficult for me to engage with Chomsky’s ideas.

I'm rather confused by this statement. I've read a number of Chomsky pieces and have listened to him speak a number of times. To say his theories were all "not true" seems, to an extent, almost impossible. Care to expand on how his theories can be taught in such a binary way?

GP may be referring to the idea that language is innate like an organ in the body/brain. The Kingdom of Speech by Tom Wolfe is a great read exploring Chomsky and other thinkers in this realm. It would have been great to see what he thought of LLMs too.

Re: Chomsky on what ChatGPT is good for (2023)

#218

Post by Workaccount2 »

jmb99 wrote: post #172
CamperBob2 wrote: post #169

Earlier quoted context omitted.

Go ask the operator of a Chinese room to do some math they weren't taught in school, and see if the translation guide helps. The analogy I've used before is a bright first-grader named Johnny. Johnny stumbles across a high school algebra book. Unless Johnny's last name is von Neumann, he isn't going to get anything out of that book. An LLM will. So much for the Chinese Room.

> Go ask the operator of a Chinese room to do some math they weren't taught in school, and see if the translation guide helps. That analogy only holds if LLMs can solve novel problems that can be proven to not exist in any form in their training material.

I think people give training data too much credit. Obviously it's important, but it also isn't a database of knowledge like it's made out to be.

You can see this in riddles that are obviously in the training set, but older or lighter models still get them wrong. Or situations where the model gets them right, but uses a different method than the ones used in the training set.

Re: Chomsky on what ChatGPT is good for (2023)

#219

Post by sdwr »

papaver-somnamb wrote: post #212

There was an interesting debate where Chomsky took a position on intelligence being rooted in symbolic reasoning and Asimov asserted a statistical foundation (ah, that was not intentional ;). LLM designs to date are purely statistical models. A pile, a morass of floating point numbers and their weighted relationships, along with the software and hardware that animates them and the user input and output that makes the…

That's not true, symbols emerge out of the statistics. Just look at the imagenet analysis that identified distinct concepts in different layers, or the experiments with ablation in LLMs.

They may not be doing strict formal logic, but they are definitely compressing information into, and operating using, symbols.

Re: Chomsky on what ChatGPT is good for (2023)

#220

Post by Workaccount2 »

xwolfi wrote: post #215
AIorNot wrote: post #187

Earlier quoted context omitted.

LLM models are to a large extent neuronal analogs of human neural architecture - of course they reason The claim of the “stochastic parrot” needs to go away Eg see: https://www.anthropic.com/news/golden-gate-claude I think the rub is that people think you need consciousness to do reasoning, I’m NOT claiming LLMs have consciousness or awareness

They are really not neuronal analogs, reasoning is far from what they do. If they reasoned, they'd stick to their guns more readily, but try to contradict an LLM and it will make any logic leap you ask it too. If you debate with me, I'll keep reasoning on the same premises and usually the difference between two humans is not in reasoning but in choice of premises. For instance you really want here to assert that LLM…

Gemini 2.5 will tell you when you're wrong
Post reply on HN