51–60 of 78 posts
The sentence subject is one thing, the sentence topic might be quite another. Consider sentences like: "He joined the not-yet-famous Liverpool band in early 1958." To many human beings the topic is quickly obvious. Parsing won't do the trick.
Earlier quoted context omitted.
What's really amazing is how old SHRDLU is. It was developed back in the late 60's, early 70's. Although... I guess you could look at it two ways: Be amazed at what SHRDLU could do in 1970, or be disappointed that, given that we had that in 1970, we don't have the "Star Trek Computer" yet in 2013.
SHRDLU is definitely amazing, especially given its age, but one's amazement is tempered a little bit (or maybe enhanced, depending on perspective) when you realize that it achieved what it did primarily through really great engineering rather that some fundamental insight about language. Since SHRDLU's world is so limited, Winograd was able to explicitly program every facet of its language understanding. Unsurprising…
Humans have good natural pattern-matching engines in their heads, but the entire body of syntax and vocabulary available to a person is the result of the memorization of a huge amount of text. I suspect the majority of people rarely ever develop truly novel words or phrases on their own (with the notable exception of Lewis Carroll). (Aside: in fact, this is exactly how "memes" work in the modern online sense; one person invents a novel word or phrase, and that is then parroted by a huge number of other people.)
I recently started work on an attempt to improve the classification of English vocabulary by grade level. I built a database using publicly-available sources, and the number of unique words that the average child has been exposed to by the 8th grade is mind boggling. One source cited 15,000 unique words and over a million words read annually.
Aside from the words themselves, children have also by that age memorized an even larger number of phrases, pieces of sentence structure, and full sentences.
I think that because we aren't able to enumerate everything we've memorized, we don't fully appreciate just how much data is stored in our heads. As a result, I think it's possible that computer science researchers have largely been chasing a ghost in terms of some kind of magical "understanding" of language; the answer to NLP might actually be to simply store and access a terabytes-sized data structure of vocabulary and phrases.
Without having Brown and NLTK in Node.js, I'm not sure how well I can add this to my port of shlomibs original code. For those who haven't seen yet, I wrote a port of the first part of this here https://github.com/jbrooksuk/node-summary Maybe later I'll give it a crack :)
Earlier quoted context omitted.
SHRDLU is definitely amazing, especially given its age, but one's amazement is tempered a little bit (or maybe enhanced, depending on perspective) when you realize that it achieved what it did primarily through really great engineering rather that some fundamental insight about language. Since SHRDLU's world is so limited, Winograd was able to explicitly program every facet of its language understanding. Unsurprising…
I think people underestimate how explicitly-programmed human language is in humans. I'm starting to think that this might be the central problem in NLP right now. Humans have good natural pattern-matching engines in their heads, but the entire body of syntax and vocabulary available to a person is the result of the memorization of a huge amount of text. I suspect the majority of people rarely ever develop truly novel…
I would be interested in this. Let me know if you plan to open this. What data sources are you using?
Earlier quoted context omitted.
SHRDLU is definitely amazing, especially given its age, but one's amazement is tempered a little bit (or maybe enhanced, depending on perspective) when you realize that it achieved what it did primarily through really great engineering rather that some fundamental insight about language. Since SHRDLU's world is so limited, Winograd was able to explicitly program every facet of its language understanding. Unsurprising…
I think people underestimate how explicitly-programmed human language is in humans. I'm starting to think that this might be the central problem in NLP right now. Humans have good natural pattern-matching engines in their heads, but the entire body of syntax and vocabulary available to a person is the result of the memorization of a huge amount of text. I suspect the majority of people rarely ever develop truly novel…
Isn't that effectively what google translate is doing? And it's results are... varied.
The method described doesn't tell you what the sentence is about it tells you which things aren't verbs and articles and does a poor job of it.
Granted single sentence keyword extraction is not easy, but this is an awful approach. You'd be much better using Word Frequency analysis to determine the rarest words in the sentence.
Earlier quoted context omitted.
SHRDLU is definitely amazing, especially given its age, but one's amazement is tempered a little bit (or maybe enhanced, depending on perspective) when you realize that it achieved what it did primarily through really great engineering rather that some fundamental insight about language. Since SHRDLU's world is so limited, Winograd was able to explicitly program every facet of its language understanding. Unsurprising…
I think people underestimate how explicitly-programmed human language is in humans. I'm starting to think that this might be the central problem in NLP right now. Humans have good natural pattern-matching engines in their heads, but the entire body of syntax and vocabulary available to a person is the result of the memorization of a huge amount of text. I suspect the majority of people rarely ever develop truly novel…
Also, after accepting the premise that humans exploit an enormous store of data in language use, there still remain very difficult questions about what kind of representations we have available, and how powerful the search and recombination mechanisms are.
Memory-based language processing exists for some time now, and while it is useful, it is certainly not the final answer to "the central problem in NLP" (whatever you define that to be, I'd suggest ambiguity resolution).
Earlier quoted context omitted.
I have the impression that the average sentence length is more on the order of 15 words per sentence, but I might be wrong. The cubic time complexity is for exhaustively finding the best parse. In practice you can use various approximation techniques, such as coarse-to-fine parsing used by the Charniak & Berkeley parsers. I believe these two are faster than the Stanford Parser, and the parameters of the approximation…
Thanks! Here is a study on average sentence length showing it at 24 words per sentence: http://ds.nahoo.net/Academic/Maths/Sentence.html In my own study, I performed analysis on a corpus of a few hundred million sentences of written text and found it at 31 words per sentence. It would be great if you can point to the use of GPU for parsing. I had never heard of the bitpar parser, will look into it.
Earlier quoted context omitted.
I think people underestimate how explicitly-programmed human language is in humans. I'm starting to think that this might be the central problem in NLP right now. Humans have good natural pattern-matching engines in their heads, but the entire body of syntax and vocabulary available to a person is the result of the memorization of a huge amount of text. I suspect the majority of people rarely ever develop truly novel…
the answer to NLP might actually be to simply store and access a terabytes-sized data structure of vocabulary and phrases. Isn't that effectively what google translate is doing? And it's results are... varied.
[1] http://code.google.com/p/android/issues/detail?id=38538 [2] http://techcrunch.com/2013/01/04/google-now-and-google-trans...
Earlier quoted context omitted.
SHRDLU is definitely amazing, especially given its age, but one's amazement is tempered a little bit (or maybe enhanced, depending on perspective) when you realize that it achieved what it did primarily through really great engineering rather that some fundamental insight about language. Since SHRDLU's world is so limited, Winograd was able to explicitly program every facet of its language understanding. Unsurprising…
I think people underestimate how explicitly-programmed human language is in humans. I'm starting to think that this might be the central problem in NLP right now. Humans have good natural pattern-matching engines in their heads, but the entire body of syntax and vocabulary available to a person is the result of the memorization of a huge amount of text. I suspect the majority of people rarely ever develop truly novel…