Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chomsky on what ChatGPT is good for (2023)

chomsky.info

Post reply on HN
Print view View original

231–240 of 389 posts

Re: Chomsky on what ChatGPT is good for (2023)

#231

Post by morsecodist »

xwolfi wrote: post #177
CamperBob2 wrote: post #169

Earlier quoted context omitted.

Go ask the operator of a Chinese room to do some math they weren't taught in school, and see if the translation guide helps. The analogy I've used before is a bright first-grader named Johnny. Johnny stumbles across a high school algebra book. Unless Johnny's last name is von Neumann, he isn't going to get anything out of that book. An LLM will. So much for the Chinese Room.

An LLM will get ... what exactly ? The ability to reorder its sentences ? The LLM doesn't think, doesn't understand, doesn't know what matters more than not, doesn't use what it learns, doesn't expand what it learns to new knowledge, doesn't enjoy reading that book and doesn't suffer through it. So what is it really gonna do with a book, that LLM ? Reorder its internal matrix to be a little bit more precise when auto…

This isn't really the meaning of the Chinese room. The Chinese room presupposes that the output is identical to that of a speaker who understands the language. It is not arguing that there is any sort of limit to what an AI can do with its output and it is compatible with the AI refusing to answer or wanting to talk about something else.

Re: Chomsky on what ChatGPT is good for (2023)

#232

Post by gf000 »

lostmsu wrote: post #226
gf000 wrote: post #109

Earlier quoted context omitted.

Though given that LLMs fundamentally can't know whether they know something or not (without a later pass of fine-tuning on what they should know) is a pretty good argument against them being good knowledge bases.

No, it is not. In mathematical limit this applies to literally everything. In practice you are not going to store video compressed with a lossless codec, for example.

Me forgetting/never having "recorded" what necklace the other person had during an important event is not at all similar to a statistical text-generation.

If they ask me the previous question I can retrospect/query my memory and tell 100% whether I know it or not - lossy compression aside. An LLM will just reply based on how likely a yes answer is with no regards to having that knowledge or not.

Re: Chomsky on what ChatGPT is good for (2023)

#233

Post by agarren »

krackers wrote: post #160
0xbadcafebee wrote: post #152

Earlier quoted context omitted.

Can LLMs actually parse human languages? Or can they react to stimuli with a trained behavioral response? Dogs can learn to sit when you say "sit", and learn to roll over when you say "roll over". But the dog doesn't parse human language; it reacts to stimuli with a trained behavioral response. (I'm not that familiar with LLM/ML, but it seems like trained behavioral response rather than intelligent parsing. I believe…

You can train LLMs on the output very complex CFGs, and it successfully learns the grammar and hierarchy needed to complete any novel prefix. This is a task much more recursive and difficult than human languages, so there's no reason to believe that LLMs aren't able to parse human languages in the formal sense as well. And of course empirically LLMs do generate valid English sentences. They may not necessarily be _co…

A “complex” cfg is still a cfg, and, giving credence to Chomsky’s hierarchy, remains computationally less complex than natural, context sensitive, grammars. Even a complex cfg can be parsed by a relatively simple program in ways that context-sensitive grammars cannot.

My understanding is that context sensitive grammars _can_ allow for recursive structures that are beyond cfgs, which is precisely why they sit below csgs in terms of computational complexity.

I don’t agree or disagree that LLMs might be, or are, capable of parsing (i.e., perception in Chomsky’s terms, or, arguably, “understanding” in any sense). But that they can learn the grammar of a “complex cfg” isn’t a convincing argument for the reasons you indicate.

Re: Chomsky on what ChatGPT is good for (2023)

#234

Post by krackers »

agarren wrote: post #233
krackers wrote: post #160

Earlier quoted context omitted.

You can train LLMs on the output very complex CFGs, and it successfully learns the grammar and hierarchy needed to complete any novel prefix. This is a task much more recursive and difficult than human languages, so there's no reason to believe that LLMs aren't able to parse human languages in the formal sense as well. And of course empirically LLMs do generate valid English sentences. They may not necessarily be _co…

A “complex” cfg is still a cfg, and, giving credence to Chomsky’s hierarchy, remains computationally less complex than natural, context sensitive, grammars. Even a complex cfg can be parsed by a relatively simple program in ways that context-sensitive grammars cannot. My understanding is that context sensitive grammars _can_ allow for recursive structures that are beyond cfgs, which is precisely why they sit below cs…

I don't think it's clear that human languages are context sensitive. The only consistent claim I can find is that at one point someone examined Swiss German and found that it's weakly context sensitive. Also empirically human language don't have that much recursion. You can artificially construct such examples, but beyond a certain depth people won't be able to parse it either.

I don't know whether the non-existence of papers studying whether LLMs can model context-sensitive grammar is because they can't, or because people haven't tested that hypothesis yet. But again empirically LLMs do seem to be able to reproduce human language just fine. The whole "hallucination" argument is precisely that LLMs are very good at reproducing the structure of language even if those statements don't encode things with the correct truth value. The fact that they successfully learn to parse complex CFGs is thus evidence that they can actually learn underlying generative mechanisms instead of simply parroting snippets of training data as naively assumed, and it's not a huge leap to imagine that they've learned some underlying "grammar" for English as well.

So if one argues that LLMs as a generative model cannot generate novel valid sentences in the English language, then that is easily falsifiable hypothesis. If we had examples of LLMs producing non-well formed sentences, people would have latched onto that by now, instead of "count Rs in strawbery" but I've never seen anyone arguing as such.

Re: Chomsky on what ChatGPT is good for (2023)

#235

Post by hyperadvanced »

Workaccount2 wrote: post #213
hyperadvanced wrote: post #186

Earlier quoted context omitted.

Reason about: sure. Independently solve novel ones without extreme amounts of guidance: I have yet to see it. Granted, for most language and programming tasks, you don’t need the latter, only the former.

99.9% of humans will never solve a novel problem. It's a bad benchmark to use here

I agree. But it’s worth being somewhat skeptical of ASI scenarios if you can’t, for example, give a well formulated math problem to a LLM and it cannot solve it. Until we get a Reimann hypothesis calculator (or equivalent for hard/old unsolved maths) it’s kind of silly to be debating the extreme ends of AI cognition theory

Re: Chomsky on what ChatGPT is good for (2023)

#236

Post by user_7832 »

loveparade wrote: post #201
atdt wrote: post #178

The level of intellectual engagement with Chomsky's ideas in the comments here is shockingly low. Surely, we are capable of holding these two thoughts: one, that the facility of LLMs is fantastic and useful, and two, that the major breakthroughs of AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution. That may change, particularly if the intel…

> AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution I would push back on this a little bit. While it has not helped us to understand our own intelligence, it has made me question whether such a thing even exists. Perhaps there are no simple and beautiful natural laws, like those that exists in Physics, that can explain how humans think and…

> The big lesson from the AI development in the last 10 years from me has been "I guess humans really aren't so special after all"

Instead I would take the opposite take.

How wonderful is it, that with naturally evolved processes and neural structures, have we been able to create what we have. Van Gogh’s paintings came out of the human brain. The Queens of the Skies - hundreds of tons of metal and composites - flying across continents in the form of a Boeing 747 or an A380 - was designed by the human brain. We went to space, have studied nature (and have conservation programs for organisms we have found to need help), took pictures the pillars of creation that are so incredibly far… all with such a “puny” structure a few cm in diameter? I think that’s freaking amazing.

Re: Chomsky on what ChatGPT is good for (2023)

#237

Post by lostmsu »

gf000 wrote: post #232
lostmsu wrote: post #226

Earlier quoted context omitted.

No, it is not. In mathematical limit this applies to literally everything. In practice you are not going to store video compressed with a lossless codec, for example.

Me forgetting/never having "recorded" what necklace the other person had during an important event is not at all similar to a statistical text-generation. If they ask me the previous question I can retrospect/query my memory and tell 100% whether I know it or not - lossy compression aside. An LLM will just reply based on how likely a yes answer is with no regards to having that knowledge or not.

You obviously forgot you previously heard about false memories and/or never thought that happens to you (would be v. ironic).

Re: Chomsky on what ChatGPT is good for (2023)

#239

Post by rf15 »

atdt wrote: post #178

The level of intellectual engagement with Chomsky's ideas in the comments here is shockingly low. Surely, we are capable of holding these two thoughts: one, that the facility of LLMs is fantastic and useful, and two, that the major breakthroughs of AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution. That may change, particularly if the intel…

> the major breakthroughs of AI this decade have not, at least so far, substantially deepened our understanding of our own intelligence and its constitution

People's illusions and willingness to debase their own authority and control to take shortcuts to optimise towards lowest effort / highest yield (not dissimilar to something you would get with... auto regressive models!) was an astonishing insight to me.

Re: Chomsky on what ChatGPT is good for (2023)

#240

Post by 1bpp »

SoftTalker wrote: post #200
CamperBob2 wrote: post #173

Earlier quoted context omitted.

They do. Spend some time using a modern reasoning model. There is a class of interesting problems, nestled between trivial ones whose answers can simply be regurgitated and difficult ones that either yield nonsense or involve tool use, that transformer networks can absolutely, incontrovertibly reason about.

Have any LLMs solved any of the big (or even lesser known) unanswered problems in math, physics, computer science? It may appear that they are solving novel problems but given the size of their training set they have probably seen them. There are very few questions a person can come up with that haven't already been asked and answered somewhere.

Google's AlphaEvolve recently produced a novel matrix multiplication function slightly faster than the previous state of the art that couldn't have been in any training data. While not a hard unsolved problem, I think it's good evidence that an LLM is capable of synthesizing new solutions to problems.
Post reply on HN