Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Get into Natural Language Processing

blog.ycombinator.com

Post reply on HN
Print view View original

21–30 of 80 posts

Re: How to Get into Natural Language Processing

#21

Post by hobofan »

fauigerzigerk wrote: post #19
hobofan wrote: post #17

I like the idea of the Paths series, though some of the points in this first article read like they could be written about most "emerging technologies". Anyway, I'm looking forward to the next one! The two questions about the PhD's do feel a little bit misplaced for a startup audience. Who here stops and thinks "Am I supposed to have a PhD to do that?", when setting out to start something new? ( )

I think a better question would be: How much math do I need to approach NLP in a way that enables me to be among the best? PhD is just an academic title and as such it is neither a necessary nor a sufficient prerequisite to approach NLP from a mathematical angle.

> How much math do I need to approach NLP in a way that enables me to be among the best?

The best in what? If you mean pushing the boundaries of research, yes, then your path there will likely involve a PhD. If you mean building the best technology products, then being able to read, understand and implement the biggest recent advances is enough and usually requires far less mathematical knowledge.

(That was intentionally written generically since I think that it applies to more than just NLP)

Re: How to Get into Natural Language Processing

#22

Post by will_pseudonym »

deegles wrote: post #6

There are a ton of libraries and tools available for NLP, so I feel that side is relatively mature. What I want are more tools for Natural Language Generation. Can anyone recommend some good ones? (beyond what's on Wikipedia)

I'm not sure what methods they use, but the "single sentence reply suggestions" created by Google's Inbox are the highest quality natural language generation that I have come across.

Re: How to Get into Natural Language Processing

#24

Post by chch »

d_burfoot wrote: post #14

> Why is NLP Hard? ... Language is highly ambiguous - it relies on subtle cues and contexts to convey meaning. This is true, but it is only part of the answer. Another part of the answer is what I call the Long Tail of Grammar. It turns out that if you try to write down all the rules of grammar, you will not get 40 or 60 rules, but something more like 100s or maybe even 1000s of rules. Most of those rules are obscure…

> or useable only in specific contexts or with specific words.

A good example of this is the Winograd Schema. You might think you can figure out a good algorithm for anaphoric resolution (i.e. If you see "Sally called and she said hello.", who is "she"?) that just relies on the structure of a sentence, without considering semantics.

But here's a counterexample:

"The city councilmen refused the demonstrators a permit because they feared violence."

Who are 'they'?

"The city councilmen refused the demonstrators a permit because they advocated violence."

Now who are 'they'?

If you're like most people, even though only the verb changed, the binding of 'they' based on the deeper semantic meaning also changed.

These sentences are called Winograd Schema[1], and there are plenty more like it.

[1] https://en.wikipedia.org/wiki/Winograd_Schema_Challenge

Re: How to Get into Natural Language Processing

#25

Post by fauigerzigerk »

hobofan wrote: post #21
fauigerzigerk wrote: post #19

Earlier quoted context omitted.

I think a better question would be: How much math do I need to approach NLP in a way that enables me to be among the best? PhD is just an academic title and as such it is neither a necessary nor a sufficient prerequisite to approach NLP from a mathematical angle.

> How much math do I need to approach NLP in a way that enables me to be among the best? The best in what? If you mean pushing the boundaries of research, yes, then your path there will likely involve a PhD. If you mean building the best technology products, then being able to read, understand and implement the biggest recent advances is enough and usually requires far less mathematical knowledge. (That was intention…

>If you mean pushing the boundaries of research, yes, then your path there will likely involve a PhD.

Yes I mean pushing the boundaries, but I think it is important to stress that a PhD is not a prerequesite to do that. Anyone who is talented enough can learn the necessary math.

Getting paid for research work is a different story of course. A PhD undoubtedly helps with that, but this is Hacker News. People might figure something out.

Re: How to Get into Natural Language Processing

#26

Post by leereeves »

d_burfoot wrote: post #14

> Why is NLP Hard? ... Language is highly ambiguous - it relies on subtle cues and contexts to convey meaning. This is true, but it is only part of the answer. Another part of the answer is what I call the Long Tail of Grammar. It turns out that if you try to write down all the rules of grammar, you will not get 40 or 60 rules, but something more like 100s or maybe even 1000s of rules. Most of those rules are obscure…

> compare the phrase "peeled peach" with "hairy-peeled peach"

Is that a rule of grammar, or simply the meanings of the adjectives "peeled" and "hairy-peeled"?

Re: How to Get into Natural Language Processing

#27

Post by danso »

Love the concept of this "How to" series. Seems like it'd be a good opportunity to spotlight the interesting HN threads on any given topic.

e.g. for NLP:

- https://news.ycombinator.com/item?id=11686029

- https://news.ycombinator.com/item?id=11690212

- https://news.ycombinator.com/item?id=1839611

Re: How to Get into Natural Language Processing

#28

Post by andrewtbham »

p1esk wrote: post #23
andrewtbham wrote: post #18

I researched deep learning for nlp for a year and compiled this list of papers and articles about some of the most interesting topics. https://github.com/andrewt3000/DL4NLP/blob/master/README.md

Have you built anything interesting?

I did some kaggle contests using image stuff.

With regards to nlp, I have a site that is using a spider to collect headlines for stocks and I have been working on clustering, sentiment analysis, and text summarization. But it's I haven't completed it.

http://www.teslanewsfeed.com/

Re: How to Get into Natural Language Processing

#29

Post by mswen »

blcArmadillo wrote: post #15

I think this is a good idea for a series. Although I think more detail needs to be given on the actual path, that is after all the purpose of the series. Most of this article seemed to be describing what NLP is and why it's hard. This isn't bad and some attention should be given to it but people looking to find the path into NLP will already be familiar with most of this information. I was expecting a bit more of a s…

Agreed on more substantive detail needed. I was surprised at the lack of mention of many of the basic techniques and domains that a person interested in should consider learning about.

The following are all germane but not mentioned: text analysis/mining, controlled vocabularies, indexing, taxonomies, ontology, semantic web, latent semantic analysis, latent dirichlet allocation, corpus analysis, document similarity analysis, tf-idf, ngrams, and skip grams just to mention a few.

In general the article is a good idea but their needs to be more of a description of the domain landscape and then "paths" plotted through that landscape that lead to interesting and useful competency.

Re: How to Get into Natural Language Processing

#30

Post by whodunser »

fnl wrote: post #16

> text summarization are examples of NLP in real-world products Can someone point me to a satisfying demo of a professional text summarization software?

The autotldr bot on reddit gets a lot of praise:

http://smmry.com (demo here)

https://np.reddit.com/r/autotldr/comments/31bfht/theory_auto...

Post reply on HN