Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How to Get into Natural Language Processing

blog.ycombinator.com

Post reply on HN
Print view View original

61–70 of 80 posts

Re: How to Get into Natural Language Processing

#61

Post by posterboy »

chch wrote: post #33
ice109 wrote: post #31

Earlier quoted context omitted.

re the council people sentences: I don't understand the problem. they're ill-defined sentences. we use heuristics to parse them but those heuristics can fail (the council denied the demonstrators permit because they feared violence... and the council was obliging). just teach the computer the heuristics like we learn them.

That's exactly the issue. The way we learn them is through world experience, which is sometimes hard to figure out how to transfer into a computer. Example: "I dropped the egg on my glass living room table and it broke!" "I dropped my hammer on my glass living room table and it broke!" These are both ill-defined semantically, but if you asked most native English speakers "what broke" for each sentence, they'd probabl…

context is everything.

"A violent mob requested a demonstration from the councilmen. The councilmen refused the permit, because they feared the violence."

I suspect, grammar begets normalization, with primary and secondary keys just like in relational databases. People are just not very good at it. EG. I'd contest the consistency of those 1000 grammar rules. Point in case, the word "violence" needs the definite article, because violence is an abstract concept (which the parent missed). All the while the indefinit and definit articles serve other purposes, eg. the quantifiers from logic (for all, there exists) which are at odds with the naive countability of the violence.

So Language is ambiguous, NLP is done probabilistic and thus is hard with at least exponential complexity.

Edit: What I mean is, the problem here is contraction omitting context. Of course databases worked before relational databases, but sometimes you really want the third normal form.

Re: How to Get into Natural Language Processing

#62

Post by andrewtbham »

leereeves wrote: post #26
d_burfoot wrote: post #14

> Why is NLP Hard? ... Language is highly ambiguous - it relies on subtle cues and contexts to convey meaning. This is true, but it is only part of the answer. Another part of the answer is what I call the Long Tail of Grammar. It turns out that if you try to write down all the rules of grammar, you will not get 40 or 60 rules, but something more like 100s or maybe even 1000s of rules. Most of those rules are obscure…

> compare the phrase "peeled peach" with "hairy-peeled peach" Is that a rule of grammar, or simply the meanings of the adjectives "peeled" and "hairy-peeled"?

yeh it seems more like another example of word sense disambiguation.

Re: How to Get into Natural Language Processing

#63

Post by hiddencost »

ktRolster wrote: post #51
hiddencost wrote: post #49

Earlier quoted context omitted.

I'm being hyperbolic, but what I'm suggesting is that grammar is a convenient fiction. The part where you speak of "how to use it" points to the break down in your thinking. (Side note: look into the difference between prescriptive and descriptive linguistics for a sense of where I'm coming from on that point.) So, you learn a new adjective. Surely you can use it like any other adjective, right? Sure. But someone can…

what I'm suggesting is that grammar is a convenient fiction. I don't think that's right either. It's not fiction, it's definitely something real.

Let me try that again: grammar is a way of describing some conventions that are often used but whose force is much weaker than almost everyone thinks. Natural language processing on the basis of grammar gets some of the most frequent uses, but immediately its limitations become extremely clear.

I think your insistence on me being precise here is a wonderful illustration of two competing approaches to language and how they make a synthesis. I'm fairly confident you know what I mean but are choosing not to accept my phrasing, so this exchange may also be an interesting illustration of how language is also the negotiation of power.

Re: How to Get into Natural Language Processing

#64

Post by impostervt »

chch wrote: post #24
d_burfoot wrote: post #14

> Why is NLP Hard? ... Language is highly ambiguous - it relies on subtle cues and contexts to convey meaning. This is true, but it is only part of the answer. Another part of the answer is what I call the Long Tail of Grammar. It turns out that if you try to write down all the rules of grammar, you will not get 40 or 60 rules, but something more like 100s or maybe even 1000s of rules. Most of those rules are obscure…

> or useable only in specific contexts or with specific words. A good example of this is the Winograd Schema. You might think you can figure out a good algorithm for anaphoric resolution (i.e. If you see "Sally called and she said hello.", who is "she"?) that just relies on the structure of a sentence, without considering semantics. But here's a counterexample: "The city councilmen refused the demonstrators a permit…

Just wanted to say thank for a great and interesting comment. Never seen the complexity of NLP summed up so well.

Re: How to Get into Natural Language Processing

#66

Post by tyingq »

deegles wrote: post #6

There are a ton of libraries and tools available for NLP, so I feel that side is relatively mature. What I want are more tools for Natural Language Generation. Can anyone recommend some good ones? (beyond what's on Wikipedia)

I tried playing around with NLG a couple of years ago. The only tool I was able to get up and running well enough to do anything was SimpleNLG[1]. It's java, not my forte, but was straightforward otherwise. Here's a basic example of use: https://github.com/simplenlg/simplenlg/wiki/Section%20V%20%E...

[1]https://github.com/simplenlg/simplenlg

Re: How to Get into Natural Language Processing

#67

Post by tyingq »

will_pseudonym wrote: post #22
deegles wrote: post #6

There are a ton of libraries and tools available for NLP, so I feel that side is relatively mature. What I want are more tools for Natural Language Generation. Can anyone recommend some good ones? (beyond what's on Wikipedia)

I'm not sure what methods they use, but the "single sentence reply suggestions" created by Google's Inbox are the highest quality natural language generation that I have come across.

I believe this is the paper that describes the approach they are using: http://www.kdd.org/kdd2016/papers/files/Paper_1069.pdf

Looks less like NLG, and more like picking existing responses from a (probably huge) corpus using ML. Hard to replicate unless you have access to the kind of data Google has.

Re: How to Get into Natural Language Processing

#68

Post by czep »

chch wrote: post #24
d_burfoot wrote: post #14

> Why is NLP Hard? ... Language is highly ambiguous - it relies on subtle cues and contexts to convey meaning. This is true, but it is only part of the answer. Another part of the answer is what I call the Long Tail of Grammar. It turns out that if you try to write down all the rules of grammar, you will not get 40 or 60 rules, but something more like 100s or maybe even 1000s of rules. Most of those rules are obscure…

> or useable only in specific contexts or with specific words. A good example of this is the Winograd Schema. You might think you can figure out a good algorithm for anaphoric resolution (i.e. If you see "Sally called and she said hello.", who is "she"?) that just relies on the structure of a sentence, without considering semantics. But here's a counterexample: "The city councilmen refused the demonstrators a permit…

This is a recognized problem and is called Word Sense Disambiguation. It's hard but not intractable. One issue is that such sentences are themselves ambiguous and even human readers may disagree on their meaning. A statistical system can make a guess based on a large corpus of word-context pairs, which can approximate what a human does when attempting to disambiguate the meaning. It won't be perfect, but again, part of this is due to the fact that the sentence as it stands alone is insufficient.

Presumably, such sentences would be contained in a paragraph that would provide additional clues as to whom the word 'they' refers. Given additional context, you could then ask, "were the protesters or the councilmen fearing violence?" Document summarization and fact extraction systems could then approximate humans in such a task.

What's interesting is that word sense ambiguity underlies a lot of comedy. For instance, "Time flies like an arrow; fruit flies like a banana." The close juxtaposition of the word "like" being used in two different contexts is what makes this sentence "funny". I think it's not too far off to say that we could eventually teach AI systems to recognize humor.

Re: How to Get into Natural Language Processing

#69

Post by visarga »

amelius wrote: post #52
visarga wrote: post #48

If you want to play with NLP, then just try Gensim, sklearn and Keras. If you're serious about NLP, it's hard stuff, you need a PHD in the field. In a way, vision is easier. Instead of discrete symbols (words) it's continuous signal which are much easier to interpret and generate from neural networks. By comparison, best language models are behind best image generation models (2-3 years behind, in my estimation). For…

So what you are saying is that a computer would be able to understand sign-language more easily than spoken language?

Computers can transcribe spoken language into text at great accuracy, but they can't understand the meaning of text at the same level of accuracy yet. Meaning is much harder than simple transcription. Voice recognition is to speech like OCR to print. What we want is to speak to computers and have them understand what we mean, like humans. Such an AI would be able to carry a conversation, extract data from and reason over documents, or perform complex actions based on verbal commands. They would need to have a good physical and conceptual understanding of the world, otherwise they could not use reasoning.

Re: How to Get into Natural Language Processing

#70

Post by ktRolster »

hiddencost wrote: post #63
ktRolster wrote: post #51

Earlier quoted context omitted.

what I'm suggesting is that grammar is a convenient fiction. I don't think that's right either. It's not fiction, it's definitely something real.

Let me try that again: grammar is a way of describing some conventions that are often used but whose force is much weaker than almost everyone thinks. Natural language processing on the basis of grammar gets some of the most frequent uses, but immediately its limitations become extremely clear. I think your insistence on me being precise here is a wonderful illustration of two competing approaches to language and how…

I'm fairly confident you know what I mean

No

Post reply on HN