Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Neuroscientists find new support for Chomsky’s “internal grammar” thesis

nyu.edu

Post reply on HN
Print view View original

61–70 of 95 posts

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#61

Post by yqx »

soheil wrote: post #38
pazimzadeh wrote: post #13

> Their results showed that the subjects’ brains distinctly tracked three components of the phrases they heard, reflecting a hierarchy in our neural processing of linguistic structures: words, phrases, and then sentences—at the same time. How are they distinguishing between grammar that's encoded in the brain due to "nature" vs grammar that's encoded in the brain via "nurture"? I think we all knew that the brain has…

Your assumption is that all information necessary and sufficient to decode grammar into its constituents is detectable from the spoken words that a child hears. According to Chomsky there is a distinction between the I-Language versus E-Language [1] (internal/ex) namely because grammar in most human languages is not the most computationally efficient way of structuring sentences hence the need for an uniquely human/i…

I don't think any assumption was made here. It is as much an assumption to say that all information necessary and sufficient to decode grammar into its constituents is not detectable from spoken words that a child is exposed to. That's a theoretical argument. Just because we can't think of a way this could be done, doesn't mean it can't be done. At the moment, we simply don't know.

I think what the comment you're responding to meant is that this study shows no evidence for an innate internal language or grammar, which I thought was Chomsky's most controversial claim.

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#62

Post by yqx »

logicallee wrote: post #17
kazinator wrote: post #12

I suspect we have a "grammar" which is based on massively parallel pattern matching over a sequence of symbols of bounded length. (I.e. why we deal with ambiguities well, but not long sentences.)

if it's so massively parallel, why do I read your comment left to right one word at a time? why garden path sentences that mislead us as we build up an erroneous parse? sure there's some fuzzy pattern matching, but lexical parsing doesn't seem so massively parallel to me - more like a huge lookup table of fuzzy matches (including whole phrases, I don't have to say much about how I'm never gonna give you up, before th…

> why do I read your comment left to right one word at a time?

I'd say because the probabilistic structure of language is such that it must be processed sequentially to be decoded correctly. The interpretation of what comes next depends on what came before. Information-theoretically, this happens to be an efficient and compact way of encoding information in the auditory medium (where input unfolds over time, rather than being presented all at once like in images).

Whether processing is "massively parallel" depends on what level of analysis you're assuming. At some level, processing in the brain is obviously massively, since that's simply how the brain's laid out.

I don't see how sequential processing bears any implication to whether or not language is or can be learned through statistical pattern matching, but I'd love to hear your angle on this.

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#63

Post by andreasvc »

YeGoblynQueenne wrote: post #50
hyperion2010 wrote: post #29

Earlier quoted context omitted.

Not only are there people who seriously suggest it, but most neuroscientists think Chomsky is completely wrong, and have quite a bit of data to back it up. The notion of innate grammers has been pretty much replaced with evidence that you get much better and more human like performance (and the ability to actually learn the bloody things) out of RNNs and the like. The interesting question is why RNNs are able to prod…

There's a misconception here: deep neural networks are used to learn grammar models, or the features for grammar models and so on, but the predominant model of natural language is still based on grammars. Google "Stanford parser". This is a state of the art natural language parser and like all parsers it relies on a grammar. The grammar it uses is a dependency grammar, a different grammar than the phrase structure gr…

The Stanford parser does both constituency (phrase-structure) and dependency parsing. Actually in the model that I know of, the dependency parses are derived from the constituency parses.

The base of the parser relies on a grammar to describe possible sentence structures, but actually most of the work in disambiguation is done using statistics, or with neural networks. The question what is and what is not grammar is rather arbitrary, there can be a continuum between simple rule-like and statistical regularities.

Ultimately the models in NLP are rarely making any kind of cognitive/neuroscientific claim about being plausible models, just effective ones. There's a specific field of computational psycholinguistics which does investigate those things.

> Also, it's really not the case that you get better performance out of neural network models of language, let alone "human like performance". We're very far from that still.

I wouldn't make such claims these days because deep learning methods are gaining ground very fast. There are many tasks at which the deep learning model is better than traditional models. Furthermore, there are already deep learning models which are better than humans at image labeling.

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#64

Post by MrQuincle »

cmarschner wrote: post #42
MrQuincle wrote: post #35

I think we also have visual grammars. We are surprised if heavy and large objects are on top of light and small objects. We have priors on lighting and shadows, depth and occlusion, inside and outside. We solve these statistically with complex priors and likelihoods. How much you can shove into the prior is the question of universal grammar. That you are born with linguistic priors is I think not so surprising. If th…

I'm not sure about that. A newborn first learns to recognize shapes, then starts exploring its environment using hand-eye coordination. It learns the rules of gravity by watching and feeling. It has a few inborn reflexes, like looking for the source of milk right after birth, but the physics engine seems very much a learnt thing. Also it seems the fundamental physics laws can easily be learnt by a system like the bra…

https://en.m.wikipedia.org/wiki/Infant_vision

I think it's a quite nice analogy. There are structures present, but they have to be trained as well. Suppose you have a hierarchical Dirichlet process as prior, you still need data to be able to define what clusters with what. And of course I won't postulate such a simple process as representing our newborn brains.

From Wikipedia: "At birth, visual structures are fully present yet immature in their potentials. From the first moment of life, there are a few innate components of an infant's visual system. Newborns can detect changes in brightness, distinguish between stationary and kinetic objects, as well as follow kinetic objects in their visual fields. However, many of these areas are very poorly developed."

Your extreme case would be that there is a priori no reason to have language processed in certain parts of the brain, like Broca's or Wernicke's regions.

It is of course possible that these are just strategically located. However, newborns with damage to these regions by oxygen deprevation at birth can already suffer from linguistic impairments.

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#65

Post by andreasvc »

vidarh wrote: post #32
logicallee wrote: post #17

Earlier quoted context omitted.

if it's so massively parallel, why do I read your comment left to right one word at a time? why garden path sentences that mislead us as we build up an erroneous parse? sure there's some fuzzy pattern matching, but lexical parsing doesn't seem so massively parallel to me - more like a huge lookup table of fuzzy matches (including whole phrases, I don't have to say much about how I'm never gonna give you up, before th…

> If it were just pattern matching that grammatical error wouldn't bother you at all even if you notice it, you would just go with the most likely fuzzy interpretation. I actually had to read the sentence about 3 times, and force myself to read it slowly word by word, before I spotted it. We often don't read one word at a time. I agree with you that "massively parallel" is probably drastically overstating it, though.

> "massively parallel" is probably drastically overstating it, though.

Think of a single-core processor executing instructions one at a time. Now think of ten billion neurons in your brain out of which a large proportion is active at any time. That is definitely massively parallel.

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#66

Post by andreasvc »

pazimzadeh wrote: post #13

> Their results showed that the subjects’ brains distinctly tracked three components of the phrases they heard, reflecting a hierarchy in our neural processing of linguistic structures: words, phrases, and then sentences—at the same time. How are they distinguishing between grammar that's encoded in the brain due to "nature" vs grammar that's encoded in the brain via "nurture"? I think we all knew that the brain has…

> How are they distinguishing between grammar that's encoded in the brain due to "nature" vs grammar that's encoded in the brain via "nurture"?

They do not.

> I think we all knew that the brain has some mechanism to detect grammar.

The press release is spinning the paper to make it sound much more controversial than it is.

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#67

Post by andreasvc »

palosanto wrote: post #48

"Neuroscientists and psychologists predominantly reject this viewpoint, contending that our comprehension does not result from an internal grammar;" Fascinating article, highly recommended. Not trying to be nitpicky but I'm going to need a citation or few to support some of the claims made about "neuroscientists and psychologists"... just saying.

No point in asking for a citation on a press release statement, go directly to the paper (which seems to be much more careful in what it claims).

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#68

Post by andreasvc »

logicallee wrote: post #17
kazinator wrote: post #12

I suspect we have a "grammar" which is based on massively parallel pattern matching over a sequence of symbols of bounded length. (I.e. why we deal with ambiguities well, but not long sentences.)

if it's so massively parallel, why do I read your comment left to right one word at a time? why garden path sentences that mislead us as we build up an erroneous parse? sure there's some fuzzy pattern matching, but lexical parsing doesn't seem so massively parallel to me - more like a huge lookup table of fuzzy matches (including whole phrases, I don't have to say much about how I'm never gonna give you up, before th…

For one both our auditory and visual senses have limited bandwidth. Reading is done in saccades, so already at the source the information comes in sentence chunks at a time.

The opportunity for massive-parallelness is in disambiguating a huge number of possible interpretations. Before you can construct the possible candidates, you first need to have processed enough of the sentence. But since you can process new sensory input at the same time as disambiguating previous input, you can get the garden path effect. What I find interesting about the garden path effect though is that it's so artificial, so the way language is used normally seems to purposely avoid ambiguities that will trip up listeners/readers.

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#69

Post by andreasvc »

hyperion2010 wrote: post #29
yongjik wrote: post #4

> Neuroscientists and psychologists predominantly reject this viewpoint, contending that our comprehension does not result from an internal grammar; rather, it is based on both statistical calculations between words and sound cues to structure. Wait, is this true? There are people who seriously suggest that we don't have an innate notion of grammar?

Not only are there people who seriously suggest it, but most neuroscientists think Chomsky is completely wrong, and have quite a bit of data to back it up. The notion of innate grammers has been pretty much replaced with evidence that you get much better and more human like performance (and the ability to actually learn the bloody things) out of RNNs and the like. The interesting question is why RNNs are able to prod…

> The notion of innate grammers has been pretty much replaced with evidence that you get much better and more human like performance (and the ability to actually learn the bloody things) out of RNNs and the like.

No not really. Barring concrete physiological evidence, innateness is simply an unfalsifiable position. I do think you can say that consensus among scientists might have shifted against innateness, but that is definitely not because of some computational model such as RNN with some passing resemblance to observed behavior of human neurons. The way these networks are configured and trained has nothing to do with real brains, nor does the abstraction in neural nets leave room for any electrochemical effects.

> The interesting question is why RNNs are able to produce things that resemble formal languages at first pass.

Is it? A neural network can achieve that by simulating a Turing machine or simpler automaton. I don't see what that brings to the table? That's just doing something we could already do, but less efficiently. I think the interesting thing is the opposite: if the RNN can do something which couldn't be done before with other models.

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#70

Post by DbauchdSloth »

YeGoblynQueenne wrote: post #44
logicallee wrote: post #17

Earlier quoted context omitted.

if it's so massively parallel, why do I read your comment left to right one word at a time? why garden path sentences that mislead us as we build up an erroneous parse? sure there's some fuzzy pattern matching, but lexical parsing doesn't seem so massively parallel to me - more like a huge lookup table of fuzzy matches (including whole phrases, I don't have to say much about how I'm never gonna give you up, before th…

>> if it's so massively parallel, why do I read your comment left to right one word at a time? For fun, I once had two friends speak to me simultaneously, one in each ear. We swapped places and did that to each other a few times. Result: nope, our language understanding unit is not parallel. You can either take in a wall of sound, OR focus to a string of words and understand its meaning. It might feel like you can al…

I am not so sure about this I suspect it is perhaps more of a learned skill. Professional DJs come to mind. To match two desperate musical tracks with differing tempo and align correctly requires following two musical tracks simultaneously. Initially it is very much as you say but over time you learn to hear them both separately and together without strain. My experience with twitch is another anecdotal counterpoint. After a lot of practice I now regularly listen to multiple audio streams at the same time and no longer hand any problem following up to three streams simultaneously these days, and it was impossible for me at first. Another issue is at what point does very fast attention switching become effectively multiprocessing? If we don't notice the context switching happening is still "happening"?
Post reply on HN