Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chomsky on what ChatGPT is good for (2023)

chomsky.info

Post reply on HN
Print view View original

181–190 of 389 posts

Re: Chomsky on what ChatGPT is good for (2023)

#181

Post by hackinthebochs »

foobarqux wrote: post #95
titzer wrote: post #46

All this interview proves is that Chomsky has fallen far, far behind how AI systems work today and is retreating to scoff at all the progress machine learning has achieved. Machine learning has given rise to AI now. It can't explain itself from principles or its architecture. But you couldn't explain your brain from principles or its architecture, you'd need all of neuroscience to do it. Because the brain is digital…

> The truth is that all of the progress on machine learning is absolutely science It is not science, which is the study of the natural world. You are using the word "science" as an honorific, meaning something like "useful technical work that I think is impressive". The reason you are so confused is that you can't distinguish studying the natural world from engineering.

LLMs certainly aren't science. But there is a "science of LLMs" going on in, e.g., the interpretability work by Anthropic.

Re: Chomsky on what ChatGPT is good for (2023)

#182

Post by numpad0 »

lanfeust6 wrote: post #52

I'm noticing that leftists overwhelmingly toe the same line on AI skepticism, which suggests to me an ideological motivation.

Leftists and intellectuals overlap a lot. LLM text must be still full of six fingered hands to many of them.

For Chomsky specifically, the entire existence of LLM, however it's framed, is a massive middle finger to him and a strike-through on a large part of his academic career. As much as I find his UG theory and its supporters irritating, it might be felt a bit unfair to someone his age.

Re: Chomsky on what ChatGPT is good for (2023)

#183

Post by AIorNot »

As much as I think of Chomsky - his linguistics approach is outside looking in, ie observational speculation compared to the last few years of LLM based tokenization semantic spaces, embedding, deep learning and mechanistic interpretation, ie:

Understanding Linguistics before LLMs:

“We think Birds fly by flapping their wings”

Understanding Linguistics Theories after LLMs:

“Understanding the physics of Aerofoils and Bernoulli’s principle mean we can replicate what birds do”

Re: Chomsky on what ChatGPT is good for (2023)

#184

Post by geysersam »

CamperBob2 wrote: post #169
GolfPopper wrote: post #168

Earlier quoted context omitted.

> Can LLMs actually parse human languages? IMHO, no, they have nothing approaching understanding. It's Chinese Rooms[1] all the way down, just with lots of bell and whistles. Spicy autocomplete. 1. https://en.wikipedia.org/wiki/Chinese_room

Go ask the operator of a Chinese room to do some math they weren't taught in school, and see if the translation guide helps. The analogy I've used before is a bright first-grader named Johnny. Johnny stumbles across a high school algebra book. Unless Johnny's last name is von Neumann, he isn't going to get anything out of that book. An LLM will. So much for the Chinese Room.

How can you make that claim? Have you ever used an LLM that hasn't encountered high school algebra in it's training data? I don't think so.

Re: Chomsky on what ChatGPT is good for (2023)

#185

Post by xwolfi »

krackers wrote: post #160
0xbadcafebee wrote: post #152

Earlier quoted context omitted.

Can LLMs actually parse human languages? Or can they react to stimuli with a trained behavioral response? Dogs can learn to sit when you say "sit", and learn to roll over when you say "roll over". But the dog doesn't parse human language; it reacts to stimuli with a trained behavioral response. (I'm not that familiar with LLM/ML, but it seems like trained behavioral response rather than intelligent parsing. I believe…

You can train LLMs on the output very complex CFGs, and it successfully learns the grammar and hierarchy needed to complete any novel prefix. This is a task much more recursive and difficult than human languages, so there's no reason to believe that LLMs aren't able to parse human languages in the formal sense as well. And of course empirically LLMs do generate valid English sentences. They may not necessarily be _co…

I disagree, I think it's clear in the article that Chomsky thinks a language also should have a human purpose.

The compression we use in languages to not label impossible adjectives against impossible nouns (green ideas is impossible as ideas don't have colors, we could have a suffix on every noun to mark what can be colored and what cannot) is because we need to transfer these over the air, and quickly, before the lion jumps on the hunter. It's one of the many attributes of "languages in the wild" (Chinese doesn't use "tenses" really, can you imagine the compressive value?), and that's what Chomsky says here:

Proceeding further with normal science, we find that the internal processes and elements of the language cannot be detected by inspection of observed phenomena. Often these elements do not even appear in speech (or writing), though their effects, often subtle, can be detected. That is yet another reason why restriction to observed phenomena, as in LLM approaches, sharply limits understanding of the internal processes that are the core objects of inquiry into the nature of language, its acquisition and use. But that is not relevant if concern for science and understanding have been abandoned in favor of other goals.

Understand what he means: you can read a million text through a machine, it will never infer why we don't label adjective and nouns to prevent confusion and "green ideas". But for us it's painfully obvious, we don't have time when we speak to do all that. And I come from a language when we label every noun with a gender, I can see how stupid and painful it is to grasp for foreigners: it doesn't make any sense. Why do we do it ? Ask ChatGPT, will it tell you that it's because we like how beautiful it all sounds, which is the stupid reason why we do that ?

Re: Chomsky on what ChatGPT is good for (2023)

#186

Post by hyperadvanced »

CamperBob2 wrote: post #173
jmb99 wrote: post #172

Earlier quoted context omitted.

> Go ask the operator of a Chinese room to do some math they weren't taught in school, and see if the translation guide helps. That analogy only holds if LLMs can solve novel problems that can be proven to not exist in any form in their training material.

They do. Spend some time using a modern reasoning model. There is a class of interesting problems, nestled between trivial ones whose answers can simply be regurgitated and difficult ones that either yield nonsense or involve tool use, that transformer networks can absolutely, incontrovertibly reason about.

Reason about: sure. Independently solve novel ones without extreme amounts of guidance: I have yet to see it.

Granted, for most language and programming tasks, you don’t need the latter, only the former.

Re: Chomsky on what ChatGPT is good for (2023)

#187

Post by AIorNot »

xwolfi wrote: post #177
CamperBob2 wrote: post #169

Earlier quoted context omitted.

Go ask the operator of a Chinese room to do some math they weren't taught in school, and see if the translation guide helps. The analogy I've used before is a bright first-grader named Johnny. Johnny stumbles across a high school algebra book. Unless Johnny's last name is von Neumann, he isn't going to get anything out of that book. An LLM will. So much for the Chinese Room.

An LLM will get ... what exactly ? The ability to reorder its sentences ? The LLM doesn't think, doesn't understand, doesn't know what matters more than not, doesn't use what it learns, doesn't expand what it learns to new knowledge, doesn't enjoy reading that book and doesn't suffer through it. So what is it really gonna do with a book, that LLM ? Reorder its internal matrix to be a little bit more precise when auto…

LLM models are to a large extent neuronal analogs of human neural architecture

- of course they reason

The claim of the “stochastic parrot” needs to go away

Eg see: https://www.anthropic.com/news/golden-gate-claude

I think the rub is that people think you need consciousness to do reasoning, I’m NOT claiming LLMs have consciousness or awareness

Re: Chomsky on what ChatGPT is good for (2023)

#188

Post by DennisP »

0xbadcafebee wrote: post #152
krackers wrote: post #142

Earlier quoted context omitted.

Restricted to linguistics, LLM's supposed lack of understanding should be a non-sequitur. If the question is whether LLMs have formed a coherent ability to parse human languages, the answer is obviously yes. In fact not just human languages, as seen with multimodality the same transformer architecture seems to work well to model and generate anything with inherent structure. I'm surprised that he doesn't mention "uni…

Can LLMs actually parse human languages? Or can they react to stimuli with a trained behavioral response? Dogs can learn to sit when you say "sit", and learn to roll over when you say "roll over". But the dog doesn't parse human language; it reacts to stimuli with a trained behavioral response. (I'm not that familiar with LLM/ML, but it seems like trained behavioral response rather than intelligent parsing. I believe…

It can have a sensible conversation with you, follow your instructions, do math and physics, and write code that performs the task you described in English. Some models can create pictures and videos matching the description you gave them, or write descriptions of a video from you.

In 2023, Microsoft released a paper saying GPT4 could do things like tell you how to stack a random collection of unrelated variously-shaped objects so they don't fall over. Things have come a long way since then.

Try out one of the advanced models, and see whether you think it understands concepts.

Re: Chomsky on what ChatGPT is good for (2023)

#189

Post by EGreg »

GolfPopper wrote: post #168
0xbadcafebee wrote: post #152

Earlier quoted context omitted.

Can LLMs actually parse human languages? Or can they react to stimuli with a trained behavioral response? Dogs can learn to sit when you say "sit", and learn to roll over when you say "roll over". But the dog doesn't parse human language; it reacts to stimuli with a trained behavioral response. (I'm not that familiar with LLM/ML, but it seems like trained behavioral response rather than intelligent parsing. I believe…

> Can LLMs actually parse human languages? IMHO, no, they have nothing approaching understanding. It's Chinese Rooms[1] all the way down, just with lots of bell and whistles. Spicy autocomplete. 1. https://en.wikipedia.org/wiki/Chinese_room

Actually, the LLMs made me realize John Searle’s “Chinese room” doesnt make much sense

Because languages have many similar concepts so the operator inside the Chinese room can understand nearly all the concepts without speaking Chinese.

And the LLM can translate to and from any language trivially, the inner layers do the actual understanding of concepts.

Re: Chomsky on what ChatGPT is good for (2023)

#190

Post by what-the-grump »

krackers wrote: post #160
0xbadcafebee wrote: post #152

Earlier quoted context omitted.

Can LLMs actually parse human languages? Or can they react to stimuli with a trained behavioral response? Dogs can learn to sit when you say "sit", and learn to roll over when you say "roll over". But the dog doesn't parse human language; it reacts to stimuli with a trained behavioral response. (I'm not that familiar with LLM/ML, but it seems like trained behavioral response rather than intelligent parsing. I believe…

You can train LLMs on the output very complex CFGs, and it successfully learns the grammar and hierarchy needed to complete any novel prefix. This is a task much more recursive and difficult than human languages, so there's no reason to believe that LLMs aren't able to parse human languages in the formal sense as well. And of course empirically LLMs do generate valid English sentences. They may not necessarily be _co…

I’ve seen ChatGPT generate bad English and I’ve seen the layer or logic / UI re-render the page as I think there is a simple spell checker that kicks in and tells the api to re-render and recheck.

I don’t believe for one second that LLMs reason, understand, know, anything.

There are plenty of times LLMs fail to generate correct sentences, and plenty of times they fail to generate correct words.

Around the time ChatGPT rolled out web search inside actions, you’d get really funky stuff back and watch other code clearly try to catch the run away.

o3 can be hot garbage if you ask it expand a specific point inside a 3 paragraph memo, the reasoning models perform very, very poorly when they are not summarizing.

There are times where the thing works like magic, other times, asking it to write me a PowerShell script that gets users by first and last name has it inventing commands that flags that don’t exist.

If the model ‘understood’, ‘followed, some sort of structure outside parroting stuff it already knows about it would be easy to spot and guide it via prompts. That is not the case even with the most advanced models today.

It’s clear that LLMs work best at specific small tasks that have a well established pattern defined in a strict language or api.

I’ve broken o3 trying to have it lift working python code, into formal python code, how? The person that wrote the code didn’t exactly code it how a developer would code a program. 140 lines of basic grab some data generate a table broke the AI and it had the ‘informal’ solution in the prompt. So no there is zero chance LMMs do more than predict.

And to be clear, it one shot a whole thing for me last night, using the GitHub/Codex/agent thing in VS code, probably saved me 30 minutes but god forbid you start from a bad / edge / poorly structured thing that doesn’t fit the mould.

Post reply on HN