Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chomsky on what ChatGPT is good for (2023)

chomsky.info

Post reply on HN
Print view View original

351–360 of 389 posts

Re: Chomsky on what ChatGPT is good for (2023)

#351

Post by foobarqux »

specialist wrote: post #327
foobarqux wrote: post #119

Earlier quoted context omitted.

I don't really understand your question but if a deep neural network predicts the weather we don't have any problem accepting that the deep neural network is not an explanatory model of the weather (the weather is not a neural net). The same is true of predicting language tokens.

Apologies, I don't know enough to articulate my question, which is probably nonsensical any way. LLMs (like GPT) and grammars (like Backus–Naur Form) are two different kinds of generative (production) systems, right? You've been (heroically) explaining Chomsky's criticism of LLMs to other noobs: grammars (theoretically) explain how humans do language, which is very different from how ChatGPT (stochastic parrots) do l…

> Since GPT mimics human language so convincingly, I've been wondering if there's any overlap of these two generative systems.

It's not really relevant if there is overlap, I'm sure you can list a bunch of ways they are similar. What's important is 1. if they are different in fundamental ways and 2. whether LLMs explain anything about the human language faculty.

For 1. the most important difference is that human languages appear to have certain constraints (roughly that language has parse tree/hierarchical structure) and (from the experiments of Moro) humans seem to not be able to learn arguably simpler structures that are not hierarchical. LLMs on the other hand can be trained on those simpler structures. That shows that the acquisition process is not the same, which is not surprising since neural networks work on arbitrary statistical data and don't have strong inductive biases.

For 2. even if it turned out that LLMs couldn't learn the same languages it doesn't explain anything. For example you could hard-code the training to fail if it detects an "impossible language" then what? You've managed to create an accurate predictor but you don't have any understanding of how or why it works. This is easier to understand with non-cognitive systems like the weather or gravity: If you create a deep neural network that accurately predicts gravity it is not the same as coming up with the general theory of relativity (which could in fact be a worse predictor for example at quantum scales). Everyone argues the ridiculous point that since LLMs are good predictors then gaining understanding about the human language faculty is useless, which is a stance that wouldn't be accepted for the study of gravity or in any other field.

Re: Chomsky on what ChatGPT is good for (2023)

#352

Post by johnisgood »

globnomulous wrote: post #349
johnisgood wrote: post #307

Earlier quoted context omitted.

I am pleasantly surprised that David Hume's writings have been mentioned. I love his works.

I have to confess this is the only essay of his I know, though it's an all-time favorite. What other Hume pieces would you recommend?

What initially drew me to David Hume was a quote from his discussions of miracles in "An Enquiry Concerning Human Understanding" (name of chapter is "Of Miracles").

That said, I began with "A Treatise of Human Nature" around the age of 17, translated to my native language (his works are not an easy read in English, IMO), due to my interest in both philosophy and psychology.

If you haven't read them yet, I would certainly recommend them. I would recommend the latter I mentioned even if you are not interested in psychology (but may be interested in epistemology, philosophy of mind, and/or ethics), as he gets into detail about his "impressions" vs "ideas".

Additionally, he is famously known for his "problem of induction" which you may already know.

Re: Chomsky on what ChatGPT is good for (2023)

#353

Post by dTal »

CamperBob2 wrote: post #169
GolfPopper wrote: post #168

Earlier quoted context omitted.

> Can LLMs actually parse human languages? IMHO, no, they have nothing approaching understanding. It's Chinese Rooms[1] all the way down, just with lots of bell and whistles. Spicy autocomplete. 1. https://en.wikipedia.org/wiki/Chinese_room

Go ask the operator of a Chinese room to do some math they weren't taught in school, and see if the translation guide helps. The analogy I've used before is a bright first-grader named Johnny. Johnny stumbles across a high school algebra book. Unless Johnny's last name is von Neumann, he isn't going to get anything out of that book. An LLM will. So much for the Chinese Room.

A "Chinese Room" absolutely will, because the original thought experiment proposed no performance limits on the setup - the Room is said to pass the Turing Test flawlessly.

People keep using "Chinese Room" to mean something it isn't and it's getting annoying. It is nothing more than a (flawed) intuition pump and should not be used as an analogy for anything, let alone LLMs. "It's a Chinese Room" is nonsensical unless there is literally an ACTUAL HUMAN in the setup somewhere - its argument, invalid as it is, is meaningless in its absence.

Re: Chomsky on what ChatGPT is good for (2023)

#354

Post by foldr »

arp242 wrote: post #315
foldr wrote: post #290

Earlier quoted context omitted.

Most likely not. This is one of his weird pieces co-authored with Jeffrey Watamull. I don’t doubt that he put his name on it voluntarily, but it reads much more like Watamull than Chomsky. The views expressed in the interview we’re commenting on are much more Chomsky-like.

He explicitly says he didn't write it in this article: "NC: Credit for the article should be given to the actual author, Jeffrey Watumull, a fine mathematician-linguist-philosopher. The two listed co-authors were consultants, who agree with the article but did not write it."

Good point! It's useful to have a reference for this, thanks. It's obvious to anyone who's read some of Chomsky's work that he didn't write the NYT article, but I can understand why others might find this claim a bit implausible, given that his name is at the top of it.

Re: Chomsky on what ChatGPT is good for (2023)

#355

Post by nz »

_glass wrote: post #280
loveparade wrote: post #201

Earlier quoted context omitted.

> AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution I would push back on this a little bit. While it has not helped us to understand our own intelligence, it has made me question whether such a thing even exists. Perhaps there are no simple and beautiful natural laws, like those that exists in Physics, that can explain how humans think and…

I think it is important to realize, that we need to understand language on our own terms. The logic of LLMs is not unlike alien technology to us. That being said, the minimalist program of Chomsky lead to nowhere, because just like programming, it found edge case after edge case, reducing it further and further, until there was no program anymore that resembled a real theory. But it is wrong to assume that the big pr…

By way of analogy, the result of the theorem prover is usually actionable (i.e. we can replace one kind of expression with its proven equivalent for some end like optimizing code-size or code-run-time), but mathematicians _still_ endeavor to translate the unwieldy and verbose machine-generated proofs into concise human-readable proofs, because those readable proofs are useful to our understanding of mathematics even long after the "productive action" has been taken.

In a way, this collaboration between the machine and the human is better than what came before, because now productive actions can be taken sooner, and mathematicians do not have to doubt whether they are searching for a proof that exists.

Re: Chomsky on what ChatGPT is good for (2023)

#356

Post by pixl97 »

Barrin92 wrote: post #272
PeterStuer wrote: post #269

Earlier quoted context omitted.

It indeed baffles me how academics overall seem so dismissive of recent breakthroughs in sub-symbolic approaches as models from which we can learn about 'intelligence'? It is as if a biochemist looks at a human brain, and concludes there is no 'intelligence' there at all, just a whole lot of electro-chemical reactions. It fully ignores the potential for emergence. Don't misunderstand me, I'm not saying 'AGI has arriv…

>It fully ignores the potential for emergence. There's two kinds of emergence, one scientific, the other a strange, vacuous notion in the absence of any theory and explanation. The first case is emergence when we for example talk about how gas or liquid states, or combustibility emerge from certain chemical or physical properties of particles. It's not just that they're emergent, we can explain how they're emergent a…

>not some magical word if you don't know how something works.

I'd disagree, emergence is typically what we don't understand. When we understand it, it's rarely considered an emergent concept, just something that is.

>They don't offer a scientific explanation.

Correct, because we don't have the tooling necessary to explain it yet. Emergence as you stated came from simpler concepts at first, for example burning hydrogen and oxygen and water emerges from that.

Ecosystems are an emergent property of living systems, ones that we can explain rather well these days after we realized there were gaps in our knowledge. It's taken millions and millions of hours of research to piece all these bits together.

Now we are at the same place in large neural nets. What you say is pointless is not pointless at all. It's pointing at the exact things we need to work on if we want to have understanding of it. But at the same time understanding isn't necessary. We have made advancements in scientific topics that we don't understand.

Re: Chomsky on what ChatGPT is good for (2023)

#357

Post by thomassmith65 »

suddenlybananas wrote: post #350
thomassmith65 wrote: post #342

Earlier quoted context omitted.

I don't have the domain knowledge to discuss that.

If you don't know what a syntactic island is, perhaps you're not the best judge of the plausibility of a linguistic theory.

Fantastic, let's have a debate about me /s

Re: Chomsky on what ChatGPT is good for (2023)

#358

Post by pixl97 »

OccamsMirror wrote: post #261
rf15 wrote: post #239

Earlier quoted context omitted.

> the major breakthroughs of AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution People's illusions and willingness to debase their own authority and control to take shortcuts to optimise towards lowest effort / highest yield (not dissimilar to something you would get with... auto regressive models!) was an astonishing insight to me.

Well said. It's wild when you think of how many "AI" products are out there that essentially entrust an LLM to make the decisions the user would otherwise make. Recruitment, trading, content creation, investment advice, medical diagnosis, legal review, dating matches, financial planning and even hiring decisions. At some point you have to wonder: is an LLM making your hiring decision really better than rolling a dice…

> it doesn't generate a neat sounding paragraph "explaining" why candidate A is the obvious choice.

Here I will argue that humans do the same thing. For any business of any size recruitment has been pretty awful in recent history. The end user, that is the manager the employee will be hired under is typically a later step after a lot of other filters, some automated some not.

At the end of the day the only way is to measure the results. Do LLMs produce better hiring results than some outside group?

Also, LLMs seem very good at medical pre-diagnosis. If you accurately portray your symptoms to them they come back with a decent list of possible candidates. In barbaric nations like the US where medical care can easily lead to bankruptcy people are going to use it as a filter to determine if they should go in for a visit.

Re: Chomsky on what ChatGPT is good for (2023)

#359

Post by mrmdp »

Chomsky has the ability to say things in a way that most laypersons of average intelligence can grasp. That is an important skill for communication of one's thoughts to the general populace.

Many of the comments herein lack that feature and seem to convey that the author might be full of him(her)self.

Also, some of the comment are a bit pejorative.

Re: Chomsky on what ChatGPT is good for (2023)

#360

Post by CamperBob2 »

dTal wrote: post #353
CamperBob2 wrote: post #169

Earlier quoted context omitted.

Go ask the operator of a Chinese room to do some math they weren't taught in school, and see if the translation guide helps. The analogy I've used before is a bright first-grader named Johnny. Johnny stumbles across a high school algebra book. Unless Johnny's last name is von Neumann, he isn't going to get anything out of that book. An LLM will. So much for the Chinese Room.

A "Chinese Room" absolutely will, because the original thought experiment proposed no performance limits on the setup - the Room is said to pass the Turing Test flawlessly. People keep using "Chinese Room" to mean something it isn't and it's getting annoying. It is nothing more than a (flawed) intuition pump and should not be used as an analogy for anything, let alone LLMs. "It's a Chinese Room" is nonsensical unless…

A Chinese Room has no attention model. The operator can look up symbolic and syntactical equivalences in both directions, English to Chinese and Chinese back to English, but they can't associate Chinese words with each other or arrive at broader inferences from doing so. An LLM can.

If I were to ask a Chinese room operator, "What would happen if gravity suddenly became half as strong while I'm drinking tea?," what would you expect as an answer?

Another question: if I were to ask "What would be an example of something a Chinese room's operator could not handle, that an actual Chinese human could?", what would you expect in response?

Claude gave me the first question in response to the second. That alone takes Chinese Rooms out of the realm of any discussion regarding LLMs, and vice versa. The thought experiment didn't prove anything when Searle came up with it, and it hasn't exactly aged well. Neither Searle nor Chomsky had any earthly idea that language was this powerful.

Post reply on HN