Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Modern language models refute Chomsky’s approach to language

scholar.google.com

Post reply on HN
Print view View original

101–110 of 244 posts

Re: Modern language models refute Chomsky’s approach to language

#101

Post by relativty »

OskarS wrote: post #71
chubot wrote: post #52

Earlier quoted context omitted.

Children raised in isolation will spontaneously develop language I don’t have a reference handy now (someone can probably do better) but I believe one way to see this is via the hearing impaired or hearing and sight impaired

The classic example which I think you’re referring to is Nicaraguan Sign Language, which developed organically in Nicaraguan schools for deaf children where neither the children nor the teachers knew any other form of sign language. It’s a fascinating story, a complex fully developed language created by children. Seems to indicate that this is indeed a very inate capability among humans in larger groups: https://en.w…

Yeah because all the other things are innate; visual/spatial awareness, touch, smell, vocalization…

Humanoids went millions of years literally learning to navigate 3D space and sense “enough heat, food, water” etc

Nomadic tribes had built shared resource depots millennia before language.

I can see the color gradients of the trees and feel muscles relax without words.

Human language beyond some utilitarian labels just instills mind viruses that bloom into delusions of grandeur.

90% of human communication is unspoken. Neuroscience shows our brains sync behavior patterns with touch and just being in a room.

Reality is full of unseen state change every moment that we have no colloquial language for; human language is hardly the source of truth and the “North star” of human society in reality.

Re: Modern language models refute Chomsky’s approach to language

#102

Post by username332211 »

AbrahamParangi wrote: post #38
steveBK123 wrote: post #17

Earlier quoted context omitted.

doesn't this make LLM a dead end towards AGI and mostly just a neat specific trick?

In order to believe this, you'd need to be able to imagine a specific test of something that an LLM could not do under any circumstances. Previously, that test could have been something like "compose a novel sonnet on a topic". Today, it is much less clear that such a test (that won't be rapidly beaten) even exists.

You could use a Markov chain to generate poetry with rhyme and meter[1]. Granted, it wouldn't be a very good one, but that just makes an LLM a refinement to older probabilistic methods.

As for something LLMs are unlikely to do under any circumstances, there's already a fairly obvious example. They can't keep a secret, hence prompt injections.

[1] https://us.pycon.org/2020/schedule/presentation/112/

Re: Modern language models refute Chomsky’s approach to language

#103

Post by norgie »

Joeri wrote: post #94
CydeWeys wrote: post #12

Earlier quoted context omitted.

> The fact that you can replicate coherent text from probabilistic analysis and modeling of a very large corpus does not mean that humans acquire and generate language the same way. Also, the LLMs are cheating! They learned from us. It's entirely possible that you do need syntax/semantics/sapience to create the original corpus, but not to duplicate it. Let's see an AlphaZero-style version of an LLM, that learns langu…

I just asked an LLM to create a language and provide a demonstration and this is what it said. Call it a stochastic parrot if you want, but I’m pretty sure a linguist can prompt it to properly invent a language. Sure, I can invent a new language for you! Let's call it "Vorin" for the purposes of this demonstration. Vorin is a tonal language with a complex system of noun classes and a relatively simple verb conjugatio…

I don't see how this proves anything. This isn't how human languages came about, there's a lot of conlang content online, and this doesn't describe a complete language anyway.

Re: Modern language models refute Chomsky’s approach to language

#104

Post by CydeWeys »

throwawaymaths wrote: post #95
CydeWeys wrote: post #77

Earlier quoted context omitted.

Humans invented English, and every other human language. Not one single human, and not in a single lifetime, but yes, we did it. LLMs are not doing this.

I mean have you seen the whole "encode a poem in made up emoji" followed by "decode the poem from emoji" stuff? I think it's not unreasonable to think with the right prompts LLMs could do this.

The LLMs that are doing this are trained on trillions of examples of human language. This is not remotely a counterexample. Now if an LLM can invent a new full language from scratch, without any training data of existing languages (like how AlphaZero learned to play Go), then that will be impressive, and a difference in kind.

Re: Modern language models refute Chomsky’s approach to language

#105

Post by famouswaffles »

mjburgess wrote: post #41
famouswaffles wrote: post #18

Earlier quoted context omitted.

>Also, the LLMs are cheating! No...they aren't. Humans aren't learning from thin air by any stretch of the imagination.

Humans learn from the structure of the world -- not the structure of language. LLMs cheat at generating text because they do so via a model of the statistical structure of text. We're in the world , it is us who stipulate the meaning of words and the structure of text. And we stipulate new meanings to novel parts of the world daily . What else is an 'iPhone' etc. ? There's nothing in `i P h o n e` which is at all lik…

>Humans learn from the structure of the world -- not the structure of language.

No we don't. Humans don't experience or perceive reality. We perceive a nice modification of it and that's after excluding all sense data points we simply aren't capable of perceiving at all.

Your brain is constantly shifting and fabricating sense data based on internal predictions and that form the basis of what you call reality. You are not learning from the structure of the world. You are learning from a simplified model of it that is fabricated at parts.

Re: Modern language models refute Chomsky’s approach to language

#106

Post by CydeWeys »

spullara wrote: post #19
CydeWeys wrote: post #12

Earlier quoted context omitted.

> The fact that you can replicate coherent text from probabilistic analysis and modeling of a very large corpus does not mean that humans acquire and generate language the same way. Also, the LLMs are cheating! They learned from us. It's entirely possible that you do need syntax/semantics/sapience to create the original corpus, but not to duplicate it. Let's see an AlphaZero-style version of an LLM, that learns langu…

Every human alive is cheating, they learned from us!

Humanity invented language though. LLMs did not; they're merely parroting our language back to us.

Re: Modern language models refute Chomsky’s approach to language

#107

Post by justrealist »

romwell wrote: post #81
justrealist wrote: post #63

Earlier quoted context omitted.

There's really no good data to support this claim.

>There's really no good data to support this claim. Fine. We know that languages did not exist at some point. We made them exist.

Yes, I acknowledge that we have evidence that groups of tribal humans will evolve language constructs over tens of thousands of years.

Whether that happens in 1 generation or 10,000 generations is completely unknown.

Re: Modern language models refute Chomsky’s approach to language

#108

Post by FabHK »

neosat wrote: post #8

The author (bafflingly) seems to have completely missed the point- since anything they state up to page 15 (at which point I stopped reading) does not refute Chomsky's points at all. The author talks about LLMs and how they generate text and then goes on to talk about how it refutes Chomsky's claim about syntax and semantics. However it does not since Chomsky's primary claim is about how HUMANS acquire language. The…

Chomsky: Birds fly by flapping their wings in a specific way while changing the angle in order to create lift and propulsion.

This paper: Planes fly, but don’t flap their wings, ergo Chomsky is wrong.

Re: Modern language models refute Chomsky’s approach to language

#109

Post by root_axis »

refulgentis wrote: post #89
somewhereoutth wrote: post #74

Earlier quoted context omitted.

Exactly. A very uncomfortable truth for those heavily invested (time/money/credence) in this latest AI wave.

It’s odd to see people doomwaving two general reasoning engines. It’s especially hard to parse a dark sweeping condemnation based on…people are investing in it? It doesn’t have the right to assign names to things? Idk what the argument is. My most charitable interpretation is “it cant reason abour anything unless we already said it” which is obviously false.

> one of which is an average 14 year old, the other an honors student college freshman

The point is that they're not those things. Yes, language models can produce solutions to language tests that a 14 year old could also produce solutions for, but a calculator can do the same thing in the dimension of math - that doesn't make a calculator a 14 year old.

Re: Modern language models refute Chomsky’s approach to language

#110

Post by JoshTko »

Language is essentially compression. And studies show humans automatically compress data relative to chimps. So there are likely biological structures that help humans compress concepts. Compression could also be an emergent property of having more layers in a NN.
Post reply on HN