Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern language models refute Chomsky’s approach to language

scholar.google.com

Post reply on HN
Print view View original

31–40 of 244 posts

Re: Modern language models refute Chomsky’s approach to language

#31

Post by cma »

api wrote: post #9
mcguire wrote: post #6

While I think some of the points in the article are interesting, the usual evidence for the Chomskian approach is the relative lack of input data for learning language by children in the wild. How much input data is used to train modern language models?

Do children really lack input data though? Human sensory input is quite a lot of data. Our languages may have common structure because that structure reflects causality and physics encountered by directly sampling the real world.

You have Hellen Keller who went blind and deaf at 19 months.

Re: Modern language models refute Chomsky’s approach to language

#32

Post by taeric »

neosat wrote: post #8

The author (bafflingly) seems to have completely missed the point- since anything they state up to page 15 (at which point I stopped reading) does not refute Chomsky's points at all. The author talks about LLMs and how they generate text and then goes on to talk about how it refutes Chomsky's claim about syntax and semantics. However it does not since Chomsky's primary claim is about how HUMANS acquire language. The…

I wouldn't be shocked to find that humans don't learn from syntax and semantics, all told. We certainly aren't doing that with our kids, as they learn. And when they start picking up language, it is rapid and impressive. Note that it comes before they can speak, too. Seeing kids ability to understand some complicated directions when they can only do rudimentary sign language is eye opening.

Re: Modern language models refute Chomsky’s approach to language

#33

Post by asdff »

neosat wrote: post #8

The author (bafflingly) seems to have completely missed the point- since anything they state up to page 15 (at which point I stopped reading) does not refute Chomsky's points at all. The author talks about LLMs and how they generate text and then goes on to talk about how it refutes Chomsky's claim about syntax and semantics. However it does not since Chomsky's primary claim is about how HUMANS acquire language. The…

Imagine being told all you need to do to learn Spanish, is to read a 300,000 word Spanish dictionary end to end so that you can probalistically come up with 1000 conversational phrases. Anyone who has learned a language can tell you it just doesn't work like that. You don't work by accumulating a massive dataset and training on it. No one can hold such a massive dataset of anything in their head at once.

Re: Modern language models refute Chomsky’s approach to language

#34

Post by kristopolous »

neosat wrote: post #8

The author (bafflingly) seems to have completely missed the point- since anything they state up to page 15 (at which point I stopped reading) does not refute Chomsky's points at all. The author talks about LLMs and how they generate text and then goes on to talk about how it refutes Chomsky's claim about syntax and semantics. However it does not since Chomsky's primary claim is about how HUMANS acquire language. The…

It's kinda of like calling a hydraulic pump "mechanical muscle".

These types of "mistakes" are more about the authors letting their intentions and hopes known on how they wish the thing to be used.

Re: Modern language models refute Chomsky’s approach to language

#35

Post by jvanderbot »

famouswaffles wrote: post #18
CydeWeys wrote: post #12

Earlier quoted context omitted.

> The fact that you can replicate coherent text from probabilistic analysis and modeling of a very large corpus does not mean that humans acquire and generate language the same way. Also, the LLMs are cheating! They learned from us. It's entirely possible that you do need syntax/semantics/sapience to create the original corpus, but not to duplicate it. Let's see an AlphaZero-style version of an LLM, that learns langu…

>Also, the LLMs are cheating! No...they aren't. Humans aren't learning from thin air by any stretch of the imagination.

Humans are, in the sense that we're working off the checkpoint passed to us from our parents: the bare minimum language centers our brain is evolved to produce regardless. The sequence of checkpoints evolves effectively from scratch, but individual humans adapt the checkpoint for their environment from birth.

I like to think of the brain as the general model we've trained with evolution, and the person's experiences as the specialization.

Re: Modern language models refute Chomsky’s approach to language

#36

Post by adastra22 »

travisgriggs wrote: post #21
steveBK123 wrote: post #17

Earlier quoted context omitted.

doesn't this make LLM a dead end towards AGI and mostly just a neat specific trick?

A better and better parrot is still a parrot then? (I’m agreeing with you basically)

Why do you think you are anything more?

Re: Modern language models refute Chomsky’s approach to language

#38

Post by AbrahamParangi »

steveBK123 wrote: post #17
MrBuddyCasino wrote: post #15

Earlier quoted context omitted.

Wordcels think LLMs imitate the human brain, when a shape rotator knows they really just imitate human language.

doesn't this make LLM a dead end towards AGI and mostly just a neat specific trick?

In order to believe this, you'd need to be able to imagine a specific test of something that an LLM could not do under any circumstances. Previously, that test could have been something like "compose a novel sonnet on a topic". Today, it is much less clear that such a test (that won't be rapidly beaten) even exists.

Re: Modern language models refute Chomsky’s approach to language

#39

Post by meepmorp »

enord wrote: post #22
famouswaffles wrote: post #18

Earlier quoted context omitted.

>Also, the LLMs are cheating! No...they aren't. Humans aren't learning from thin air by any stretch of the imagination.

We did, at least once.

No, humans learn their languages through observation and generation - watching older people speak, and then imitating it. They get corrections, too, when they mispronounce or misuse words.

Re: Modern language models refute Chomsky’s approach to language

#40

Post by janalsncm »

shaunxcode wrote: post #20
neosat wrote: post #8

The author (bafflingly) seems to have completely missed the point- since anything they state up to page 15 (at which point I stopped reading) does not refute Chomsky's points at all. The author talks about LLMs and how they generate text and then goes on to talk about how it refutes Chomsky's claim about syntax and semantics. However it does not since Chomsky's primary claim is about how HUMANS acquire language. The…

Thank you! It is like arguing that a human engaging in the creation of a landscape portrait using the traditional method of oil painting has been "refuted" by a computer generating vector graphics from statistical descriptions of the same scene. Both yield art but they are clearly different in interesting ways. Neither refutes or outmodes the other. Or maybe I'm wrong and actually trees have refuted mushrooms!

Since large fungi came first, maybe trees have refuted the mushrooms?

https://en.m.wikipedia.org/wiki/Prototaxites

Post reply on HN