Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chomsky on what ChatGPT is good for (2023)

chomsky.info

Post reply on HN
Print view View original

361–370 of 389 posts

Re: Chomsky on what ChatGPT is good for (2023)

#361

Post by foobarqux »

xron wrote: post #303
atdt wrote: post #178

The level of intellectual engagement with Chomsky's ideas in the comments here is shockingly low. Surely, we are capable of holding these two thoughts: one, that the facility of LLMs is fantastic and useful, and two, that the major breakthroughs of AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution. That may change, particularly if the intel…

Chompsky's central criticism of LLMs is that they can learn impossible languages just as easily as they learn possible languages. He refers to this repeatedly in the linked interview. Therefore, they cannot teach us about our own intelligence. However, a paper published last year (Mission: Impossible Language Models, Kallini et al.) proved that LLMs do NOT learn impossible languages as easily as they learn possible l…

The authors of that paper misunderstand what "impossible languages" refers to. It doesn't refer to any language a human can't learn. It refers to computationally simple plausible alternative languages that humans can't learn, in particular linear-order (non-hierarchical structure) languages.

Re: Chomsky on what ChatGPT is good for (2023)

#362

Post by lamp_book »

atdt wrote: post #178

The level of intellectual engagement with Chomsky's ideas in the comments here is shockingly low. Surely, we are capable of holding these two thoughts: one, that the facility of LLMs is fantastic and useful, and two, that the major breakthroughs of AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution. That may change, particularly if the intel…

In my opinion it will or already has redefined our conceptual models of intelligence - just like physical models of atoms or gravitational mechanics evolved and newer models replace the older. The older models aren't invalidated (all models are wrong, after all), but their limits are better understood.

People are waiting for this Prometheus-level moment with AI where it resembles us exactly but exceeds our capabilities, but I don't think that's necessary. It parallels humanity explaining Nature in our own image as God and claiming it was the other way around.

Re: Chomsky on what ChatGPT is good for (2023)

#363

Post by dTal »

CamperBob2 wrote: post #360
dTal wrote: post #353

Earlier quoted context omitted.

A "Chinese Room" absolutely will, because the original thought experiment proposed no performance limits on the setup - the Room is said to pass the Turing Test flawlessly. People keep using "Chinese Room" to mean something it isn't and it's getting annoying. It is nothing more than a (flawed) intuition pump and should not be used as an analogy for anything, let alone LLMs. "It's a Chinese Room" is nonsensical unless…

A Chinese Room has no attention model. The operator can look up symbolic and syntactical equivalences in both directions, English to Chinese and Chinese back to English, but they can't associate Chinese words with each other or arrive at broader inferences from doing so. An LLM can. If I were to ask a Chinese room operator, "What would happen if gravity suddenly became half as strong while I'm drinking tea?," what wo…

Where are you getting all this (wrong) detail about the internals of the Chinese Room? The thought experiment merely says that the operator consults "books" and follows "instructions" (no doubt Turing-complete but otherwise unspecified) for manipulating symbols they they explicitly DO NOT understand - they do NOT have access to "symbolic and syntactical equivalences" - that is the POINT of the thought experiment. But the instructions in the books in a Chinese Room could perfectly well have an attention model. The details are irrelevant, because - I stress again - Searle's Chinese Room is not cognitively limited, by definition. Its hypothetical output is indistinguishable from a Chinese human.

I tend to agree that Chinese Rooms should be kept out of LLM discussions. In addition to it being a flawed thought experiment, of all the dozens of times I've seen them brought up, not a single example has demonstrated understanding of what a Chinese Room is anyway.

Re: Chomsky on what ChatGPT is good for (2023)

#364

Post by CamperBob2 »

dTal wrote: post #363
CamperBob2 wrote: post #360

Earlier quoted context omitted.

A Chinese Room has no attention model. The operator can look up symbolic and syntactical equivalences in both directions, English to Chinese and Chinese back to English, but they can't associate Chinese words with each other or arrive at broader inferences from doing so. An LLM can. If I were to ask a Chinese room operator, "What would happen if gravity suddenly became half as strong while I'm drinking tea?," what wo…

Where are you getting all this (wrong) detail about the internals of the Chinese Room? The thought experiment merely says that the operator consults "books" and follows "instructions" (no doubt Turing-complete but otherwise unspecified) for manipulating symbols they they explicitly DO NOT understand - they do NOT have access to "symbolic and syntactical equivalences" - that is the POINT of the thought experiment. But…

The details are irrelevant, because - I stress again - Searle's Chinese Room is not cognitively limited, by definition.

So said Searle. But without specifying what he meant, it was a circular statement at best. Punting to "it passes a Turing Test" just turns it into a different debate about a different flawed test.

The operator has no idea what he's doing. He doesn't know Chinese. He has a Borges-scale library of Chinese books and a symbol-to-symbol translation guide. He can do nothing but manipulate symbols he doesn't understand. How anyone can pass a well-administered Turing test without state retention and context-based reflection, I don't know, but we've already put more thought into this than Searle did.

Re: Chomsky on what ChatGPT is good for (2023)

#365

Post by _glass »

suddenlybananas wrote: post #310
_glass wrote: post #280

Earlier quoted context omitted.

I think it is important to realize, that we need to understand language on our own terms. The logic of LLMs is not unlike alien technology to us. That being said, the minimalist program of Chomsky lead to nowhere, because just like programming, it found edge case after edge case, reducing it further and further, until there was no program anymore that resembled a real theory. But it is wrong to assume that the big pr…

>That being said, the minimalist program of Chomsky lead to nowhere, because just like programming, it found edge case after edge case, reducing it further and further, until there was no program anymore that resembled a real theory As someone who has worked in linguistics, I don't really see what you're talking about. Minimalism is not full of exceptions (please elaborate on a specific example if you have one). Mini…

Yes, and the project can be criticised by reducing until there's no value anymore. Well known instances of this process:

- Predicate Fronting in Free Relatives: In sentences like “What John saw was a surprise,” labeling the fronted predicate is not without problems, Merge doesn’t yield a clear head.

- Optional Verb Movement in Persian: Yes-no questions where verbs can optionally move (e.g., “Did you go?” vs. “You went?”) messes up feature-checking’s binary mode.

- Non-Matching Free Relatives with Pied-Piping: Structures like “In whichever city you live, you’ll find culture” mess up standard labeling, needs extra stipulations.

- Some Subjects in Finnish: Nominative vs. non-nominative subjects (e.g., “Minua kylmä” [me-ACC cold]) complicate that Minimalist case assignment.

Re: Chomsky on what ChatGPT is good for (2023)

#366

Post by Groxx »

CamperBob2 wrote: post #243
Groxx wrote: post #202

Earlier quoted context omitted.

I have at least encountered many LLMs with many school's worth of algebra knowledge, but fail miserably at algebra problems. Similarly, they've ingested human-centuries or more of spelling bee related text, but can't reliably count the number of Rs in strawberry. (yes, I understand tokenization is to blame for a large part of this. perhaps that kind of limitation applies to other things too? )

Similarly, they've ingested human-centuries or more of spelling bee related text, but can't reliably count the number of Rs in strawberry Sigh

That sigh might be a chronic condition, if it's happening even when people demonstrate a decent understanding of the causes. You may want to get that looked at.

Re: Chomsky on what ChatGPT is good for (2023)

#367

Post by AfterHIA »

atdt wrote: post #178

The level of intellectual engagement with Chomsky's ideas in the comments here is shockingly low. Surely, we are capable of holding these two thoughts: one, that the facility of LLMs is fantastic and useful, and two, that the major breakthroughs of AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution. That may change, particularly if the intel…

What exactly do you mean, "analogous to our own" and, "in a deep way" without making an appeal to magic or non-yet discovered fields of science? I understand what you're saying but when you scrutinize these things you end up in a place that's less scientific than one might think. That kind of seems to be one of Chomsky's salient points; we really, really need to get a handle on when we're doing science in the contemporary Kuhnian sense and philosophy.

The AI works on English, C++, Smalltalk, Klingon, nonsense, and gibberish. Like Turing's paper this illustrates the difference between, "machines being able to think" and, "machines being able to demonstrate some well understood mathematical process like pattern matching."

https://en.wikipedia.org/wiki/Computing_Machinery_and_Intell...

Re: Chomsky on what ChatGPT is good for (2023)

#368

Post by qwery »

calibas wrote: post #99

The fact that we have figured out how to translate language into something a computer can "understand" should thrill linguists. Taking a word (token) and abstracting it's "meaning" as a 1,000-dimension vector seems like something that should revolutionize the field of linguistics. A whole new tool for analyzing and understanding the underlying patterns of all language! And there's a fact here that's very hard to disp…

Why would that thrill linguists? I'm not saying it hasn't/wouldn't/shouldn't, but I don't see why this technology would have the dramatic impact you imagine.

Is/was the same true for ASCII/Smalltalk/binary? They are all another way to translate language into something the computer "understands".

Perhaps the fact that it hasn't would lead some to question the validity of their claims. When a scientist makes a claim about how something works, it's expected that they prove it.

If the technology is as you say, show us.

Re: Chomsky on what ChatGPT is good for (2023)

#369

Post by qwery »

visarga wrote: post #286

Brains don't have innate grammar more than languages are selected to fit baby brains. Chomsky got it backwards, languages co-evolved with human brains to fit our capacities and needs. If a language is not useful or can't be learned by children, it does not expand, it just disappears. It's like wondering how well your shoes fit your feet, forgetting that shoes are made and chosen to fit your feet in the first place.

You could say that languages developed ("evolved") to fit the indisputable human biological faculty for language.

Re: Chomsky on what ChatGPT is good for (2023)

#370

Post by DarknessFalls »

papaver-somnamb wrote: post #212

There was an interesting debate where Chomsky took a position on intelligence being rooted in symbolic reasoning and Asimov asserted a statistical foundation (ah, that was not intentional ;). LLM designs to date are purely statistical models. A pile, a morass of floating point numbers and their weighted relationships, along with the software and hardware that animates them and the user input and output that makes the…

I think we are ignoring that the statistical aspect of our ability to reason effectively and to apply logic was predicated on the deaths of millions of our ancestors. When they made the wrong decision, they likely didn't reproduce. When they made the right decision, that particular configuration of their cortical substrate was carried forward a generation. The product of this cross-generational training could have easily led to non-intelligence, and often does, but we have survivor's bias in our favor.
Post reply on HN