Fun problem: Write a parser for the English language. See it fail at tweets :)
I dont to need to write a program to do that.
How to Get into Natural Language Processing
11–20 of 80 posts
Re: How to Get into Natural Language Processing
#12NLP right now looks like the computer vision 5 years ago: DL methods are starting to work really well, so a lot of "traditional" methods to process text might soon become obsolete.
The goal is to just feed gigabytes of raw text to a huge, complex neural network, and hope it will extract relevant features.
Re: How to Get into Natural Language Processing
#13One of the best places to start is reading this patent from Berkeley Lab/DOE which word2vec was based on https://www.google.com/patents/US7987191
What insight does the patent offer, which you won't get from reading the word2vec papers?
Re: How to Get into Natural Language Processing
#14> Why is NLP Hard? ... Language is highly ambiguous - it relies on subtle cues and contexts to convey meaning.
This is true, but it is only part of the answer.
Another part of the answer is what I call the Long Tail of Grammar. It turns out that if you try to write down all the rules of grammar, you will not get 40 or 60 rules, but something more like 100s or maybe even 1000s of rules. Most of those rules are obscure, rare, archaic, or useable only in specific contexts or with specific words. However, they are part of the language, a native speaker will be able to use them and comprehend them without difficulty, and an NLP system must be able to "understand" them in order to extract the correct meaning from a sentence.
As just a minor example off the top of my head, compare the phrase "peeled peach" with "hairy-peeled peach". The former phrase means a peach without a peel, while the latter means a peach with a hairy peel. So a good NLP system must not only recognize the existence of the two grammatical rules, but also be able to disambiguate them correctly.
Re: How to Get into Natural Language Processing
#15I think this is a good idea for a series. Although I think more detail needs to be given on the actual path, that is after all the purpose of the series. Most of this article seemed to be describing what NLP is and why it's hard. This isn't bad and some attention should be given to it but people looking to find the path into NLP will already be familiar with most of this information. I was expecting a bit more of a syllabus type format. There was mention of needing some college level algebra and statics, I would have liked more detail in this area with links to more resources (classes, articles, datasets, etc). Keep up the good work!
Re: How to Get into Natural Language Processing
#16> text summarization are examples of NLP in real-world products
Can someone point me to a satisfying demo of a professional text summarization software?
Re: How to Get into Natural Language Processing
#17I like the idea of the Paths series, though some of the points in this first article read like they could be written about most "emerging technologies". Anyway, I'm looking forward to the next one!
The two questions about the PhD's do feel a little bit misplaced for a startup audience. Who here stops and thinks "Am I supposed to have a PhD to do that?", when setting out to start something new? ()
Re: How to Get into Natural Language Processing
#18I researched deep learning for nlp for a year and compiled this list of papers and articles about some of the most interesting topics.
Re: How to Get into Natural Language Processing
#19I like the idea of the Paths series, though some of the points in this first article read like they could be written about most "emerging technologies". Anyway, I'm looking forward to the next one! The two questions about the PhD's do feel a little bit misplaced for a startup audience. Who here stops and thinks "Am I supposed to have a PhD to do that?", when setting out to start something new? ( )
I think a better question would be: How much math do I need to approach NLP in a way that enables me to be among the best?
PhD is just an academic title and as such it is neither a necessary nor a sufficient prerequisite to approach NLP from a mathematical angle.
Re: How to Get into Natural Language Processing
#20> Why is NLP Hard? ... Language is highly ambiguous - it relies on subtle cues and contexts to convey meaning. This is true, but it is only part of the answer. Another part of the answer is what I call the Long Tail of Grammar. It turns out that if you try to write down all the rules of grammar, you will not get 40 or 60 rules, but something more like 100s or maybe even 1000s of rules. Most of those rules are obscure…
Not just that, grammar rules can come into existence/change spontaneously. Here's an example and name for this phenomenon:
Because syntactic change.