Neuroscientists find new support for Chomsky’s “internal grammar” thesis • Hacker News Board

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#71

Post by andreasvc » Wed, Dec 09, 2015, 2:19 PM UTC

saurik wrote: post #7

yongjik wrote: post #4

> Neuroscientists and psychologists predominantly reject this viewpoint, contending that our comprehension does not result from an internal grammar; rather, it is based on both statistical calculations between words and sound cues to structure. Wait, is this true? There are people who seriously suggest that we don't have an innate notion of grammar?

I am having a frustrating time finding a good summary of the position, but the branch that studies this is informally known as West Coast Linguistics.

That's just one example, construction grammar (not limited to US west coast). The innateness of "universal grammar" can be argued against on evolutionary grounds (how could one universal grammar evolve and end up in the DNA of all humans?), neuroscientifically (how can you encode a fixed grammar when everyone's brain is different and continually changing?), from comparative linguistics (there are many languages and they can be radically different), etc.

If it's not about innateness but about the claims of "internal grammar" from the press release then it because less clear that there would be many people who would reject such a position. I think this is a misleading on the part of the press release. Whether something was innate or learned, I think most people would agree that knowledge of language is internalized. So then the question remains whether you call it grammar or something else, and whether statistics plays a part should have no bearing on that if you ask me.

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#72

Post by EmiliLoenne » Wed, Dec 09, 2015, 2:26 PM UTC

cma wrote: post #53

cmarschner wrote: post #42

Earlier quoted context omitted.

I'm not sure about that. A newborn first learns to recognize shapes, then starts exploring its environment using hand-eye coordination. It learns the rules of gravity by watching and feeling. It has a few inborn reflexes, like looking for the source of milk right after birth, but the physics engine seems very much a learnt thing. Also it seems the fundamental physics laws can easily be learnt by a system like the bra…

In experiments, a newborn of a few hours will reflexively dodge/cower from a quickly growing shape circle on a screen (something is approaching and about to collide!). It has no experience at that point of being struck by any moving object or anything like that.

Or the experimenter interprets any reflex according to his theory.

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#73

Post by lawpoop » Wed, Dec 09, 2015, 2:28 PM UTC

cmarschner wrote: post #42

MrQuincle wrote: post #35

I think we also have visual grammars. We are surprised if heavy and large objects are on top of light and small objects. We have priors on lighting and shadows, depth and occlusion, inside and outside. We solve these statistically with complex priors and likelihoods. How much you can shove into the prior is the question of universal grammar. That you are born with linguistic priors is I think not so surprising. If th…

I'm not sure about that. A newborn first learns to recognize shapes, then starts exploring its environment using hand-eye coordination. It learns the rules of gravity by watching and feeling. It has a few inborn reflexes, like looking for the source of milk right after birth, but the physics engine seems very much a learnt thing. Also it seems the fundamental physics laws can easily be learnt by a system like the bra…

Since most babies tend to follow the steps of development according to a timeline (i.e. a, then b, then c, and never out of order), and typically on time, isn't this rather evidence that such neural system grow more than the physics system is learned?

The idea that some complex systems can simply grow without inputs is not unfounded in the biological world -- witness, for example, grazing animals such as cows and giraffes that plop out of the birth canal, shakily stand up, and being trotting around. They couldn't have trained that system in the womb.

Similarly, human babies who are growing complex neural systems over years might look like they are learning, but instead they are just growing. That's not to say that we don't learn things later on-- certainly we do-- but that's not evidence that babies are necessarily learning all the skills they become capable of in toddlerhood.

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#74

Post by lawpoop » Wed, Dec 09, 2015, 2:52 PM UTC

logicallee wrote: post #17

kazinator wrote: post #12

I suspect we have a "grammar" which is based on massively parallel pattern matching over a sequence of symbols of bounded length. (I.e. why we deal with ambiguities well, but not long sentences.)

if it's so massively parallel, why do I read your comment left to right one word at a time? why garden path sentences that mislead us as we build up an erroneous parse? sure there's some fuzzy pattern matching, but lexical parsing doesn't seem so massively parallel to me - more like a huge lookup table of fuzzy matches (including whole phrases, I don't have to say much about how I'm never gonna give you up, before th…

The thing is, people don't speak they way they are taught to write, in complete sentences. Instead, they speak _in actual conversation_ in a series of fragments, which often don't end properly (we trail off, let the other complete our thought, interject, etc), and would be ungrammatical if sequenced together. Remember, language evolved to communicate with others in conversation, not to deliver speeches and text monologues.

Consider this text: "Well the whole point of it was... I mean, when I was in school, we often-- sometimes we would go hunting after school, and I mean, you know -- look, nothing like this ever-- we could never have imagined it."

That's 5 or 6 sentence fragments, one after another. If you processed it in a naively serial way, you just end up with one big ungrammatical sentence. I'm only able to clarify it with punctuation (mdashes and commas) because I could parse it correctly.

A parallel system would benefit not from parsing such a string, but creating a set of candidate interpretations, (all the words together, this is a series of three phrases, this is a series of five phrases, etc) and determining the most meaningful match.

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#75

Post by DavidSJ » Wed, Dec 09, 2015, 3:15 PM UTC

lawpoop wrote: post #73

cmarschner wrote: post #42

Earlier quoted context omitted.

I'm not sure about that. A newborn first learns to recognize shapes, then starts exploring its environment using hand-eye coordination. It learns the rules of gravity by watching and feeling. It has a few inborn reflexes, like looking for the source of milk right after birth, but the physics engine seems very much a learnt thing. Also it seems the fundamental physics laws can easily be learnt by a system like the bra…

Since most babies tend to follow the steps of development according to a timeline (i.e. a, then b, then c, and never out of order), and typically on time, isn't this rather evidence that such neural system grow more than the physics system is learned? The idea that some complex systems can simply grow without inputs is not unfounded in the biological world -- witness, for example, grazing animals such as cows and gir…

Artificial neural networks also go through predictable and repeatable stages of development as they train on a corpus.

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#76

Post by sawwit » Wed, Dec 09, 2015, 4:05 PM UTC

MrQuincle wrote: post #35

I think we also have visual grammars. We are surprised if heavy and large objects are on top of light and small objects. We have priors on lighting and shadows, depth and occlusion, inside and outside. We solve these statistically with complex priors and likelihoods. How much you can shove into the prior is the question of universal grammar. That you are born with linguistic priors is I think not so surprising. If th…

I wouldn't find it too surprising if the brain would abstract actions and actors, perhaps even the tense from thoughts and sensory input and map them to auditive features and motor programs for (sub)vocalization and the internal monologue. I think, visual thinking is different from that. Mental rotation and visualizing objects have no actors and actions in it. It is just recall of movement, shape and texture patterns that have been previously encountered. However, I think visual thinking is often enhanced by thinking in language. The auditive memory can be leveraged as an additional buffer and as a means of verifying and constructing visual thoughts.

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#77

Post by YeGoblynQueenne » Wed, Dec 09, 2015, 4:32 PM UTC

andreasvc wrote: post #63

YeGoblynQueenne wrote: post #50

Earlier quoted context omitted.

There's a misconception here: deep neural networks are used to learn grammar models, or the features for grammar models and so on, but the predominant model of natural language is still based on grammars. Google "Stanford parser". This is a state of the art natural language parser and like all parsers it relies on a grammar. The grammar it uses is a dependency grammar, a different grammar than the phrase structure gr…

The Stanford parser does both constituency (phrase-structure) and dependency parsing. Actually in the model that I know of, the dependency parses are derived from the constituency parses. The base of the parser relies on a grammar to describe possible sentence structures, but actually most of the work in disambiguation is done using statistics, or with neural networks. The question what is and what is not grammar is…

>> I wouldn't make such claims these days because deep learning methods are gaining ground very fast. There are many tasks at which the deep learning model is better than traditional models. Furthermore, there are already deep learning models which are better than humans at image labeling.

We've been here before. Back in the Olden Days of GOFAI, expert systems used to routinely outperform human experts at all sorts of cognitive tasks (medical diagnosis being a typical example). There was a huge amount of excitement and people promising wild things were just around the corner. A few years down the line, there's the same excitement around a completely different technology and expert systems are nowhere to be seen. So I'll keep my expectations at about mid-range and wait for another ten years before I say I know exactly what's going on.

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#78

Post by mladenkovacevic » Wed, Dec 09, 2015, 4:33 PM UTC

lawpoop wrote: post #73

cmarschner wrote: post #42

Earlier quoted context omitted.

I'm not sure about that. A newborn first learns to recognize shapes, then starts exploring its environment using hand-eye coordination. It learns the rules of gravity by watching and feeling. It has a few inborn reflexes, like looking for the source of milk right after birth, but the physics engine seems very much a learnt thing. Also it seems the fundamental physics laws can easily be learnt by a system like the bra…

Since most babies tend to follow the steps of development according to a timeline (i.e. a, then b, then c, and never out of order), and typically on time, isn't this rather evidence that such neural system grow more than the physics system is learned? The idea that some complex systems can simply grow without inputs is not unfounded in the biological world -- witness, for example, grazing animals such as cows and gir…

The way you formulated that is fascinating and made me understand the whole debate a little better. The strict order in which we grasp things does indeed indicate that these basic skills are perhaps not left to chance waiting until the correct input is provided, but are hardwired into the process that develops our brains.

I guess the Chomsky debate is whether grammar is one of those basic skills. Would two children left unsupervised by adults through their formative years construct their own language that surpasses the communication exhibited by gorillas for example?

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#79

Post by drumdance » Wed, Dec 09, 2015, 4:34 PM UTC

DbauchdSloth wrote: post #70

YeGoblynQueenne wrote: post #44

Earlier quoted context omitted.

>> if it's so massively parallel, why do I read your comment left to right one word at a time? For fun, I once had two friends speak to me simultaneously, one in each ear. We swapped places and did that to each other a few times. Result: nope, our language understanding unit is not parallel. You can either take in a wall of sound, OR focus to a string of words and understand its meaning. It might feel like you can al…

I am not so sure about this I suspect it is perhaps more of a learned skill. Professional DJs come to mind. To match two desperate musical tracks with differing tempo and align correctly requires following two musical tracks simultaneously. Initially it is very much as you say but over time you learn to hear them both separately and together without strain. My experience with twitch is another anecdotal counterpoint.…

This rings true to me. As a trained musician, I know that I hear music differently from most untrained people. I notice the bass line, or the way the melody inverts, or anticipate the end of a phrase.

Several years ago Kathy Sierra wrote a great post about how app developers should try to educate users from the first time they use the product and "raise the resolution" on how they think about the problem the app is designed to solve.

http://headrush.typepad.com/creating_passionate_users/2005/1...

Re: Neuroscientists find new support for Chomsky’s “internal grammar” thesis

#80

Post by YeGoblynQueenne » Wed, Dec 09, 2015, 4:40 PM UTC

DbauchdSloth wrote: post #70

YeGoblynQueenne wrote: post #44

Earlier quoted context omitted.

>> if it's so massively parallel, why do I read your comment left to right one word at a time? For fun, I once had two friends speak to me simultaneously, one in each ear. We swapped places and did that to each other a few times. Result: nope, our language understanding unit is not parallel. You can either take in a wall of sound, OR focus to a string of words and understand its meaning. It might feel like you can al…

I am not so sure about this I suspect it is perhaps more of a learned skill. Professional DJs come to mind. To match two desperate musical tracks with differing tempo and align correctly requires following two musical tracks simultaneously. Initially it is very much as you say but over time you learn to hear them both separately and together without strain. My experience with twitch is another anecdotal counterpoint.…

>> Another issue is at what point does very fast attention switching become effectively multiprocessing? If we don't notice the context switching happening is still "happening"?

Well, OK, that's one big question.

However, I can process music and language at the same time. I think I can process multiple pieces of music at the same time, for example I can follow a complex piece of classical music or jazz with multiple instruments playing at various signatures and so on (full disclosure: I'm a drummer). I can't quite do that with speech though. So maybe music and language are just different things and they're processed in different ways?