Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Get into Natural Language Processing

blog.ycombinator.com

Post reply on HN
Print view View original

41–50 of 80 posts

Re: How to Get into Natural Language Processing

#41

Post by [deleted] »

chch wrote: post #33
ice109 wrote: post #31

Earlier quoted context omitted.

re the council people sentences: I don't understand the problem. they're ill-defined sentences. we use heuristics to parse them but those heuristics can fail (the council denied the demonstrators permit because they feared violence... and the council was obliging). just teach the computer the heuristics like we learn them.

That's exactly the issue. The way we learn them is through world experience, which is sometimes hard to figure out how to transfer into a computer. Example: "I dropped the egg on my glass living room table and it broke!" "I dropped my hammer on my glass living room table and it broke!" These are both ill-defined semantically, but if you asked most native English speakers "what broke" for each sentence, they'd probabl…

[deleted]

Re: How to Get into Natural Language Processing

#42

Post by ramshanker »

chch wrote: post #33
ice109 wrote: post #31

Earlier quoted context omitted.

re the council people sentences: I don't understand the problem. they're ill-defined sentences. we use heuristics to parse them but those heuristics can fail (the council denied the demonstrators permit because they feared violence... and the council was obliging). just teach the computer the heuristics like we learn them.

That's exactly the issue. The way we learn them is through world experience, which is sometimes hard to figure out how to transfer into a computer. Example: "I dropped the egg on my glass living room table and it broke!" "I dropped my hammer on my glass living room table and it broke!" These are both ill-defined semantically, but if you asked most native English speakers "what broke" for each sentence, they'd probabl…

Reading this post, I get the feel that none of the AI training could be as good as humans until we train all human perceivable aspects together. An entire human brain equivalent.

OR are we just well-designed-auto-trained Neural network. :D

Re: How to Get into Natural Language Processing

#44

Post by ktRolster »

hiddencost wrote: post #40
throwaway91111 wrote: post #37

Earlier quoted context omitted.

This seems like a great argument for automatic grammar learning. How far has current research taken us in that direction? I'd much rather feed it edge cases to accommodate than rules to follow.

They tried that in the 70s and 80s until they realized they were wasting their time. Probabilistic context free grammars were a thing, also. Long story short, there's no such thing as grammar but they're a nice fiction for talking about communication. The deeper you get into NLP the more you (1) see what jelinek was talking about when he said "every time I fire a linguist, accuracy goes up" (he was hiring physicists…

there's no such thing as grammar

I'm not sure what you mean by this. Certainly adjectives are a thing, and if I learn a new adjective, "anguilliform" for example, I have never heard that in context, but I know exactly how to use it. That is grammar, right?

Re: How to Get into Natural Language Processing

#45

Post by rsivapr »

whodunser wrote: post #30
fnl wrote: post #16

> text summarization are examples of NLP in real-world products Can someone point me to a satisfying demo of a professional text summarization software?

The autotldr bot on reddit gets a lot of praise: http://smmry.com (demo here) https://np.reddit.com/r/autotldr/comments/31bfht/theory_auto...

Worth mentioning here, the text summarization engines are pretty much only usable on news articles or text with clean paragraph structure (yes, non-tech literature works too). Popular text summarization tools as they stand now, fails on anything else.

Re: How to Get into Natural Language Processing

#46

Post by vincentschen »

mswen wrote: post #29
blcArmadillo wrote: post #15

I think this is a good idea for a series. Although I think more detail needs to be given on the actual path, that is after all the purpose of the series. Most of this article seemed to be describing what NLP is and why it's hard. This isn't bad and some attention should be given to it but people looking to find the path into NLP will already be familiar with most of this information. I was expecting a bit more of a s…

Agreed on more substantive detail needed. I was surprised at the lack of mention of many of the basic techniques and domains that a person interested in should consider learning about. The following are all germane but not mentioned: text analysis/mining, controlled vocabularies, indexing, taxonomies, ontology, semantic web, latent semantic analysis, latent dirichlet allocation, corpus analysis, document similarity a…

That's a great point. I wonder if there would be a better way to introduce meaningful, actionable topics of study to an introductory-level audience of people who may have never heard of NLP.

Re: How to Get into Natural Language Processing

#47

Post by vincentschen »

fauigerzigerk wrote: post #25
hobofan wrote: post #21

Earlier quoted context omitted.

> How much math do I need to approach NLP in a way that enables me to be among the best? The best in what? If you mean pushing the boundaries of research, yes, then your path there will likely involve a PhD. If you mean building the best technology products, then being able to read, understand and implement the biggest recent advances is enough and usually requires far less mathematical knowledge. (That was intention…

>If you mean pushing the boundaries of research, yes, then your path there will likely involve a PhD. Yes I mean pushing the boundaries, but I think it is important to stress that a PhD is not a prerequesite to do that. Anyone who is talented enough can learn the necessary math. Getting paid for research work is a different story of course. A PhD undoubtedly helps with that, but this is Hacker News. People might figu…

Perhaps the distinction between "analysts" and "builders" would be better context for discussing math background in startups than PhDs.

Re: How to Get into Natural Language Processing

#48

Post by visarga »

If you want to play with NLP, then just try Gensim, sklearn and Keras. If you're serious about NLP, it's hard stuff, you need a PHD in the field.

In a way, vision is easier. Instead of discrete symbols (words) it's continuous signal which are much easier to interpret and generate from neural networks. By comparison, best language models are behind best image generation models (2-3 years behind, in my estimation).

For example, there are few applications of GANs to text, and many applications to images, GANs being the hottest thing in deep learning now. So you have to keep in mind that NLP is by and large still not solved. There is no decent conversational chat bot yet. We can reason over small pieces of text but that is far from full understanding. NLP at this level is hard.

What you can easily do now is to classify text, detect sentiment, entities, word vectors, grammatical parsing and summarization. All are low level stuff.

Re: How to Get into Natural Language Processing

#49

Post by hiddencost »

ktRolster wrote: post #44
hiddencost wrote: post #40

Earlier quoted context omitted.

They tried that in the 70s and 80s until they realized they were wasting their time. Probabilistic context free grammars were a thing, also. Long story short, there's no such thing as grammar but they're a nice fiction for talking about communication. The deeper you get into NLP the more you (1) see what jelinek was talking about when he said "every time I fire a linguist, accuracy goes up" (he was hiring physicists…

there's no such thing as grammar I'm not sure what you mean by this. Certainly adjectives are a thing, and if I learn a new adjective, "anguilliform" for example, I have never heard that in context, but I know exactly how to use it. That is grammar, right?

I'm being hyperbolic, but what I'm suggesting is that grammar is a convenient fiction. The part where you speak of "how to use it" points to the break down in your thinking.

(Side note: look into the difference between prescriptive and descriptive linguistics for a sense of where I'm coming from on that point.)

So, you learn a new adjective. Surely you can use it like any other adjective, right? Sure. But someone can also very comfortably use it in a way that violates your notions of how adjectives work, and you'll probably know what they mean. Or you'll use it in a context that makes sense to you, but not to someone you're talking to. The way to use language is in the way that allows others to understand you. Paradoxically that doesn't actually mean adhering to some arbitrary set of rules. Here's an important paper [0]. Basic idea: you get a small community isolated, and they just rip the rules to shreds, but what they end up building is often much higher bandwidth and allows for more complex ideas.

The point I'm pushing on is this: engineers especially think of language as an agreed upon set of rules which can be used correctly or incorrectly. Turns out that in practice it's a chaotic mess of individuals who abuse the rules mercilessly with minimal regard for how they're supposed to use it, and still get along happily. Developing an understanding of the inherent fuzziness of words and structure in communication can actually help a person develop significant capacity for self expression.

The failure to understand this is one of the reasons most engineers write shitty poetry. :P

[0] http://onlinelibrary.wiley.com/doi/10.1111/1467-9481.00177/f...

Re: How to Get into Natural Language Processing

#50

Post by WildUtah »

will_pseudonym wrote: post #20
d_burfoot wrote: post #14

> Why is NLP Hard? ... Language is highly ambiguous - it relies on subtle cues and contexts to convey meaning. This is true, but it is only part of the answer. Another part of the answer is what I call the Long Tail of Grammar. It turns out that if you try to write down all the rules of grammar, you will not get 40 or 60 rules, but something more like 100s or maybe even 1000s of rules. Most of those rules are obscure…

Not just that, grammar rules can come into existence/change spontaneously. Here's an example and name for this phenomenon: Because syntactic change.

That isn't really a rule change. The lexicon just added a new preposition, 'because,' to the old list. It means almost the same thing as the old preposition 'because of.'

There's a difference between coining a new word and changing a grammar rule.

Post reply on HN