Live data from Hacker News

What Happened to Old School NLP?

languagengine.co

41–49 of 49 posts

Re: What Happened to Old School NLP?

#41

Earlier quoted context omitted.

> The problem is slippery though, people always reuse words to express different ideas : > "We can't buy any bread because we haven't got any bread." As long as we are talking about syntactic parsing, this is not a problem as long as the attachment is the same. In both cases 'bread' is the direct object of the main verb. Of course, there are cases where a particular word can be used both as a direct object and a subj…

I think they're saying tho that in one case "bread" means a kind of food, and in the other, it means money. If the different uses were more directly connected to particular words that could disambiguate, it wouldn't be too hard -- head features typically can do this -- but the disambiguation here is far more conceptual in original. You know that you need money to buy things, and so you know that not having money is a…

Ah, also I just remembered the thing I was "quoting"

Rick: Why haven't we got any bread?

Neil: Because we haven't got any bread.

Re: What Happened to Old School NLP?

#42

Earlier quoted context omitted.

I think they're saying tho that in one case "bread" means a kind of food, and in the other, it means money. If the different uses were more directly connected to particular words that could disambiguate, it wouldn't be too hard -- head features typically can do this -- but the disambiguation here is far more conceptual in original. You know that you need money to buy things, and so you know that not having money is a…

Ah, also I just remembered the thing I was "quoting" Rick: Why haven't we got any bread? Neil: Because we haven't got any bread.

The Young Ones?

Re: What Happened to Old School NLP?

#44
post #40
post #21

This is a pretty good explanation of why almost all practical applications of NLP are now accomplished by statistics rather than fancy linguistic grammar models you might have read about in a Chomsky book. Old school NLP has always fascinated me though, and I'm pretty excited about what might be possible in the future by using more than purely statistical methods for accomplishing NLP tasks. Maybe the author could ha…

It's important to make a distinction between (i) Chomskyan linguistics, (ii) 90s style symbolic systems, (iii) 90s/early 2000s style statistical systems and (iv) 2010s style statistical systems. Chomskyan linguistics assumes that statistics and related stuff is not relevant at all, and that instead you need to find the god-given (or at least innate) Universal Grammar and then everything will be great. 90s style symbo…

Actually this isn't true, wrt Chomsky. Chomskyan linguistics assumes statistics is very important (and this has been noted by Chomsky himself since at least the early 1960s). Chomsky simply argues that statistics is insufficient on its own. And in truth, most NLPers believe this, but they rarely admit it. Most/all NLP projects have some form of "universal grammar", tho usually its something like a regular grammar (~ a Markov chain) or at best a probabilistic CFG (PCFG). I suspect the reason is that, to some extent, hierarchical structures like this seem so natural that its hard to imagine what else you could do, so there's a tendency to co treat CFGs as not even a grammar choice, but it is. There are other kinds of grammars (such as pregroup grammars) which lack these notions of hierarchy but work perfectly well for the same domains as CFGs, just in very different ways.

Re: What Happened to Old School NLP?

#45

I really liked the article and some of the Blog-Headlines seemed to be interesting as well. But try as I might, I was not able to find a rss/atom/xml-Feed for plugging this ressource into my feedreader. Sadly, so I will probably miss upcoming interesting posts.

I'm working on it, have no fear! I'm hand coding the blog right now because I don't want to use a CMS, I really don't like the options I have. I might just hand-code an RSS feed.

Re: What Happened to Old School NLP?

#47

I really liked the article and some of the Blog-Headlines seemed to be interesting as well. But try as I might, I was not able to find a rss/atom/xml-Feed for plugging this ressource into my feedreader. Sadly, so I will probably miss upcoming interesting posts.

I'm working on it, have no fear! I'm hand coding the blog right now because I don't want to use a CMS, I really don't like the options I have. I might just hand-code an RSS feed.

Try a static blog generator like hugo or octopress...

Re: What Happened to Old School NLP?

#48
post #40

Earlier quoted context omitted.

It's important to make a distinction between (i) Chomskyan linguistics, (ii) 90s style symbolic systems, (iii) 90s/early 2000s style statistical systems and (iv) 2010s style statistical systems. Chomskyan linguistics assumes that statistics and related stuff is not relevant at all, and that instead you need to find the god-given (or at least innate) Universal Grammar and then everything will be great. 90s style symbo…

Actually this isn't true, wrt Chomsky. Chomskyan linguistics assumes statistics is very important (and this has been noted by Chomsky himself since at least the early 1960s). Chomsky simply argues that statistics is insufficient on its own. And in truth, most NLPers believe this, but they rarely admit it. Most/all NLP projects have some form of "universal grammar", tho usually its something like a regular grammar (~…

Quoting a symposium with Chomsky talking about statistical AI: http://languagelog.ldc.upenn.edu/myl/PinkerChomskyMIT.html

"I think there have been some successes, but a lot of failures. The successes, to my knowledge at least, are those that integrate statistical analysis with some universal grammar properties, some fundamental properties of language; when they're integrated, you sometimes get results [...] On the other hand there's a lot work which tries to do sophisticated statistical analysis, you know bayesian and so on and so forth, without any concern for the uh actual structure of language, as far as I'm aware that only achieves success in a very odd sense of success. There is a notion of success which has developed in computational cognitive science in recent years which I think is novel in the history of science. It interprets success as approximating unanalyzed data."

The model that has become dominant in statistical AI -- positing a basic grammar that is strongly underconstrained and eliminating spurious analyses not through "universal grammar" (i.e. presupposed innate structures) but through learned parameters, would be something that Chomsky has been very much against; Simultaneously, work that models grammar with enough precision that you could derive predictions from it (e.g. Ed Stabler's grammar implementation) are seen as nice-to-have but not central to the undertaking of generative grammar.

And I think Chomsky put his thumb right on the difference in goals: Chomsky defines progress in linguistics as work that posits the right ("universal") structures, and argues that these are cognitively real and innate, whereas statistical AI is more interested in predicting useful things with structures that may or may not correspond to anything that is cognitively real.

To people nowadays, the whole notion of constrained "universal" models with few statistics versus underconstrained "statistical" models seems to be a very minor one, since today's statistical models have a lot of structure, and people doing generative grammar aren't totally opposed to using statistics or optimality theory to select most-plausible structures. But, back in the day, when the most expressive statistical models people used were HMMs [hidden Markov model - a probabilistic regular grammar] and PCFGs [probabilistic context-free grammars], the gap was much wider, whereas nowadays the models are a bit more similar while the goals are still different.

Re: What Happened to Old School NLP?

#49
post #47

Earlier quoted context omitted.

I'm working on it, have no fear! I'm hand coding the blog right now because I don't want to use a CMS, I really don't like the options I have. I might just hand-code an RSS feed.

Try a static blog generator like hugo or octopress...

yeah, people have suggested a variety of things. i'll add those to my list of static generators to investigate, thanks :)
Post reply on HN