Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Chomsky on what ChatGPT is good for (2023)

chomsky.info

Post reply on HN
Print view View original

91–100 of 389 posts

Re: Chomsky on what ChatGPT is good for (2023)

#91

Post by ogogmad »

cj wrote: post #81
titzer wrote: post #46

All this interview proves is that Chomsky has fallen far, far behind how AI systems work today and is retreating to scoff at all the progress machine learning has achieved. Machine learning has given rise to AI now. It can't explain itself from principles or its architecture. But you couldn't explain your brain from principles or its architecture, you'd need all of neuroscience to do it. Because the brain is digital…

> Not to mention all the "just engineering" of making chips crunch incredible amounts of numbers. Are LLM's still the same black box as they were described as a couple years ago? Are their inner workings at least slightly better understood than in the past? Running tens of thousands of chips crunching a bajillion numbers a second sounds fun, but that's not automatically "engineering". You can have the same chips crun…

> Running tens of thousands of chips crunching a bajillion numbers a second sounds fun, but that's not automatically "engineering".

Not if some properties are unexpectedly emergent. Then it is science. For instance, why should a generic statistical model be able to learn how to fill in blanks in text using a finite number of samples? And why should a generic blank-filler be able to produce a coherent chat bot that can even help you write code?

Some have even claimed that statistical modelling shouldn't able to produce coherent speech, because it would need impossible amounts of data, or the optimisation problem might be too hard, or because of Goedel's incompleteness theorem somehow implying that human-level intelligence is uncomputable, etc. The fact that we have a talking robot means that those people were wrong. That should count as a scientific breakthrough.

Re: Chomsky on what ChatGPT is good for (2023)

#93

Post by foobarqux »

0xDEAFBEAD wrote: post #53

I confess my opinion of Noam Chomsky dropped a lot from reading this interview. The way he set up a "Tom Jones" strawman and kept dismissing positions using language like "we'd laugh", "total absurdity", etc. was really disappointing. I always assumed that academics were only like that on reddit, and in real life they actually made a serious effort at rigorous argument, avoiding logical fallacies and the like. Yet he…

"Tom Jones" isn't a strawman, Chomsky is addressing an actual argument in a published paper from Steven Piantadosi. He's using a pseudonym to be polite and not call him out by name.

> instead of even attempting to summarize the arguments for his position..

He makes a very clear, simple argument, accessible to any layperson who can read. If you are studying insects what you are interested in is how insects do it not what other mechanisms you can come up with to "beat" insects. This isn't complicated.

Re: Chomsky on what ChatGPT is good for (2023)

#94

Post by A4ET8a8uTh0_v2 »

It is unfortunate opinion, because I personally hold Chomsky in fairly high regard and give most of his thoughts I am familiar with a reasonable amount of consideration if only because he could, I suppose in the olden days now, articulate his points well and make you question your own thought process. This no longer seems to be the case though as I found the linked article somewhat difficult to follow. I suppose age can get to anyone.

Not that I am an LLM zealot. Frankly, some of the clear trajectory it puts humans on makes me question our futures in this timeline. But even if I am not a zealot, but merely an amused, but bored middle class rube, the serious issues with it ( privacy, detailed personal profiling that surpasses existing systems, energy use, and actual power of those who wield it ), I can see it being implemented everywhere with a mix of glee and annoyance.

I know for a fact it will break things and break things hard and it will be people, who know how things actually work that will need to fix those.

I will be very honest though. I think Chomsky is stuck in his internal model of the world and unable to shake it off. Even his arguments fall flat, because they don't fit the domain well. It seems like they should given that he practically made his name on syntax theory ( which suggests his thoughts should translate well into it ) and yet.. they don't.

I have a minor pet theory on this, but I am still working on putting it into some coherent words.

Re: Chomsky on what ChatGPT is good for (2023)

#95

Post by foobarqux »

titzer wrote: post #46

All this interview proves is that Chomsky has fallen far, far behind how AI systems work today and is retreating to scoff at all the progress machine learning has achieved. Machine learning has given rise to AI now. It can't explain itself from principles or its architecture. But you couldn't explain your brain from principles or its architecture, you'd need all of neuroscience to do it. Because the brain is digital…

> The truth is that all of the progress on machine learning is absolutely science

It is not science, which is the study of the natural world. You are using the word "science" as an honorific, meaning something like "useful technical work that I think is impressive".

The reason you are so confused is that you can't distinguish studying the natural world from engineering.

Re: Chomsky on what ChatGPT is good for (2023)

#96

Post by lanfeust6 »

Supermancho wrote: post #59
lanfeust6 wrote: post #52

I'm noticing that leftists overwhelmingly toe the same line on AI skepticism, which suggests to me an ideological motivation.

> AI skepticism Isn't AI optimism an ideological motivation? It's a spectrum, not a mental model.

Whether one expects AI to be powerful or weak should have nothing to do with political slant, but here it seems to inform the opinion. It begs the question: what do they want to be true? The enemy is both too strong and too weak.

They're firmly on one extreme end of the spectrum. I feel as though I'm somewhere in between.

Re: Chomsky on what ChatGPT is good for (2023)

#97

Post by [deleted] »

netcan wrote: post #26

Insect behaviour. Flight of birds. Turtle navigation. A footballer crossing the field to intercept a football. This is what Chomsky always wanted ai to be... especially language ai. Clever solutions to complex problems. Simple once you know how they work. Elegant. I sympathize. I'm a curious human. We like elegant, simple revelations that reveal how out complex world is really simple once you know it's secrets. This…

[deleted]

Re: Chomsky on what ChatGPT is good for (2023)

#98

Post by Shorel »

ogogmad wrote: post #91
cj wrote: post #81

Earlier quoted context omitted.

> Not to mention all the "just engineering" of making chips crunch incredible amounts of numbers. Are LLM's still the same black box as they were described as a couple years ago? Are their inner workings at least slightly better understood than in the past? Running tens of thousands of chips crunching a bajillion numbers a second sounds fun, but that's not automatically "engineering". You can have the same chips crun…

> Running tens of thousands of chips crunching a bajillion numbers a second sounds fun, but that's not automatically "engineering". Not if some properties are unexpectedly emergent. Then it is science. For instance, why should a generic statistical model be able to learn how to fill in blanks in text using a finite number of samples? And why should a generic blank-filler be able to produce a coherent chat bot that ca…

> because it would need impossible amounts of data

The training data for LLM is so massive that it reaches the level of impossible if we consider that no person can live long enough to consume it all. Or even a small percent of it.

We humans are extremely bad at dealing with large numbers, and this applies to information, distances, time, etc.

Re: Chomsky on what ChatGPT is good for (2023)

#99

Post by calibas »

The fact that we have figured out how to translate language into something a computer can "understand" should thrill linguists. Taking a word (token) and abstracting it's "meaning" as a 1,000-dimension vector seems like something that should revolutionize the field of linguistics. A whole new tool for analyzing and understanding the underlying patterns of all language!

And there's a fact here that's very hard to dispute, this method works. I can give a computer instructions and it "understands" them in a way that wasn't possible before LLMs. The main debate now is over the semantics of words like "understanding" and whether or not an LLM is conscious in the same way as a human being (it isn't).

Re: Chomsky on what ChatGPT is good for (2023)

#100

Post by ogogmad »

Shorel wrote: post #98
ogogmad wrote: post #91

Earlier quoted context omitted.

> Running tens of thousands of chips crunching a bajillion numbers a second sounds fun, but that's not automatically "engineering". Not if some properties are unexpectedly emergent. Then it is science. For instance, why should a generic statistical model be able to learn how to fill in blanks in text using a finite number of samples? And why should a generic blank-filler be able to produce a coherent chat bot that ca…

> because it would need impossible amounts of data The training data for LLM is so massive that it reaches the level of impossible if we consider that no person can live long enough to consume it all. Or even a small percent of it. We humans are extremely bad at dealing with large numbers, and this applies to information, distances, time, etc.

Your final remark sounds condescending. Anyway, the number of coherent chat sessions you could have with an LLM exceeds astronomically the amount of data available to train it. How is that even possible?
Post reply on HN