Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

An Efficient Way to Extract the Main Topics from a Sentence

thetokenizer.com

Post reply on HN
Print view View original

41–50 of 78 posts

Re: An Efficient Way to Extract the Main Topics from a Sentence

#41

Post by shmageggy »

mindcrime wrote: post #30
tannerc wrote: post #19

Earlier quoted context omitted.

Why does this seem so scary/amazing? It's basically taking what computers already do really well today and adding a language "mask" onto it. Yet it still blows me away.

What's really amazing is how old SHRDLU is. It was developed back in the late 60's, early 70's. Although... I guess you could look at it two ways: Be amazed at what SHRDLU could do in 1970, or be disappointed that, given that we had that in 1970, we don't have the "Star Trek Computer" yet in 2013.

SHRDLU is definitely amazing, especially given its age, but one's amazement is tempered a little bit (or maybe enhanced, depending on perspective) when you realize that it achieved what it did primarily through really great engineering rather that some fundamental insight about language. Since SHRDLU's world is so limited, Winograd was able to explicitly program every facet of its language understanding. Unsurprisingly, this approach is totally not scalable and this reveals a little about why we don't have fully human-like language programs.

Re: An Efficient Way to Extract the Main Topics from a Sentence

#42

Post by mindcrime »

shmageggy wrote: post #41
mindcrime wrote: post #30

Earlier quoted context omitted.

What's really amazing is how old SHRDLU is. It was developed back in the late 60's, early 70's. Although... I guess you could look at it two ways: Be amazed at what SHRDLU could do in 1970, or be disappointed that, given that we had that in 1970, we don't have the "Star Trek Computer" yet in 2013.

SHRDLU is definitely amazing, especially given its age, but one's amazement is tempered a little bit (or maybe enhanced, depending on perspective) when you realize that it achieved what it did primarily through really great engineering rather that some fundamental insight about language. Since SHRDLU's world is so limited, Winograd was able to explicitly program every facet of its language understanding. Unsurprising…

Since SHRDLU's world is so limited, Winograd was able to explicitly program every facet of its language understanding. Unsurprisingly, this approach is totally not scalable and this reveals a little about why we don't have fully human-like language programs.

That's a good point. It does lead one to wonder, however, if techniques inspired to SHRDLU could (or do) have application in domain-specific applications where the world is likewise restricted. Given the increases in raw horsepower available since SHURDLU was first developed, I find myself wondering if we couldn't do some pretty useful things today, using this approach.

Re: An Efficient Way to Extract the Main Topics from a Sentence

#43

Post by jweese »

mindcrime wrote: post #42
shmageggy wrote: post #41

Earlier quoted context omitted.

SHRDLU is definitely amazing, especially given its age, but one's amazement is tempered a little bit (or maybe enhanced, depending on perspective) when you realize that it achieved what it did primarily through really great engineering rather that some fundamental insight about language. Since SHRDLU's world is so limited, Winograd was able to explicitly program every facet of its language understanding. Unsurprising…

Since SHRDLU's world is so limited, Winograd was able to explicitly program every facet of its language understanding. Unsurprisingly, this approach is totally not scalable and this reveals a little about why we don't have fully human-like language programs. That's a good point. It does lead one to wonder, however, if techniques inspired to SHRDLU could (or do) have application in domain-specific applications where t…

Yes. For example, consider interlingual machine translation. Most systems today (like Google) use statistical MT that learns patterns from millions of examples. In interlingua, by contrast, you analyze the input sentence to form a language-independent representation of the sentence's meaning. Then you use that representation to generate a sentence in a new language.

As you might expect, this is basically impossible for wide-domain MT because we don't have unambiguous representations of the meaning of every sentence, and we don't necessarily know how to combine them, and there's a lot of non-compositional phrases, and on and on.

However, if we restrict ourselves to one small domain, interlingua can work. For example, the KANT system [1] is an interlingua that is built for translating technical manuals for Caterpillar products (bulldozers and so on). The input has to be written in a restricted subset of English (Caterpillar Technical English), but then you can analyze it exactly with hand-written rules, and produce exact output in the target language.

[1] http://www2.lti.cs.cmu.edu/Research/Kant/

Re: An Efficient Way to Extract the Main Topics from a Sentence

#44

Post by stonemetal »

danso wrote: post #14
arrrg wrote: post #7

Earlier quoted context omitted.

Nah, it filtered out the meaningless buzzwords. It's a dashboard for online content. That pretty much implies the picking and finding of said online content to be displayed on the dashboard.

Huh? "Curating" and "discovering" may be overused tech verbs, but they are vital in describing what the "dashboard" does. For example, you would never describe the Google Analytics dashboard as something that curates or discovers. And far worse than buzzwords are adjectives. Does "beautiful" add anything to that sentence?

I disagree. Dashboards curate by definition. Some car dashboards display have a tachometer, some don't it depends on the car's focus. Google Analytics similarly doesn't display all available information, just that which will be useful. Dashboards always display a limited subset of available information that makes sense to the current context aka curated.

Discovering is also vacuous. The whole point of a dashboard is to convey information. Do I discover how fast I am going by looking at my car's dashboard? Do I discover my website's traffic by going to Google analytics? Sure, I wouldn't use them if I didn't get the information I need from them. So using an online content dashboard that doesn't deliver online content of some sort would be a waste of time.

Beautiful adds something because not all dashboards are beautiful.

Re: An Efficient Way to Extract the Main Topics from a Sentence

#45

Post by microtonal »

syllogism wrote: post #33
microtonal wrote: post #25

Earlier quoted context omitted.

Accuracy is state-of-the-art -- 92-93% depending on the beam width and the evaluation set (Stanford or MALT dependencies). I assume that this is for English? A former colleague of mine compared two statistical dependency parsers (Malt and MST) to a rule-based dependency parser with a maxent disambiguation model, for Dutch. The rule-based system outperforms the statistical dependency parsers by a wide margin, both in-…

Yes, for English. There's a standard multi-lingual evaluation for statistical dependency parsers (the CoNLL 2007 data), but none for constituency parsers. I had a look at that paper, but didn't read it carefully. All I can really say is that there's a real evaluation problem between rule-based and statistical parsers. Rule-based parsers recover richer representations, but tend to have lower coverage over arbitrary da…

Rule-based parsers recover richer representations, but tend to have lower coverage over arbitrary data

This parser usually has a coverage around ~95-97%. In one experiment we also parsed Flemish (which uses some constructions that wouldn't be considered grammatical in Dutch) and obtained a coverage of ~94%.

It also has a robustness component that attempts to provide an analysis for as many constituents as possible if no fully spanning parse can be found.

Having worked on the CCG formalism through my PhD, I can say that even small differences in annotation scheme can make a big difference in which parsers come out ahead.

Certainly. But you are not mentioning the other elephant in the room: Dutch is a free word order language and also permits very liberal ordering in the middle field. The rule-based grammar may benefit from the detailed constraints in the lexical attribute-value structures.

Re: An Efficient Way to Extract the Main Topics from a Sentence

#46

Post by syllogism »

bravura wrote: post #40
syllogism wrote: post #16

Earlier quoted context omitted.

The Stanford parser is particularly slow --- it's in java, and it's written for research more than anything. The C&C CCG parser runs at about 60-80 sentences a second, although it gives either CCG constituents or dependencies -- so the output may take some interpretation. Shift-reduce dependency parsers are linear time, and are giving state-of-the-art results. My parser's currently a pain in the ass to install, as it…

The Stanford parser is particularly slow --- it's in java, and it's written for research more than anything. The choice of language is not what causes the slowness of the Stanford parser. It's the choice of search strategy, which trades-off speed for accuracy. Shift-reduce dependency parsers are linear time, and are giving state-of-the-art results. No, this is incorrect. The choice of parsing logic (shift-reduce, dep…

> The choice of language is not what causes the slowness of the Stanford parser. It's the choice of search strategy, which trades-off speed for accuracy.

Even amongst chart parsers it's not a very quick implementation.

> The choice of parsing logic (shift-reduce, dependency) and the search strategy (greedy, sometimes erroneously called "deterministic") are orthogonal. It's the greedy search strategy that leads to linear time performance.

They're really only conceptually orthogonal, because in practice once you choose shift-reduce you're always going to choose greedy/deterministic/whatever search. I'm not aware of any shift-reduce/transition-based parsing results that don't use 1-best or beam search.

> I am not aware of state-of-the-art results from greedy shift-reduce parsers. Do you mind sharing?

Zhang and Nivre (2011) http://www.sutd.edu.sg/cmsresource/faculty/yuezhang/acl11j.p... 93.5 UAS on Stanford basic dependencies. This is the best published result on the dataset, and the best published dependency parsing result for English.

Probably the parser is still worse than the C&J reranking parser, when 200 parses are supplied to the reranker. But I'd be very surprised if it wasn't better than the Stanford parser, and probably also the Berkeley parser.

Re: An Efficient Way to Extract the Main Topics from a Sentence

#47

Post by unlucky »

grecy wrote: post #38
teeja wrote: post #23

The sentence subject is one thing, the sentence topic might be quite another. Consider sentences like: "He joined the not-yet-famous Liverpool band in early 1958." To many human beings the topic is quickly obvious. Parsing won't do the trick.

Ask anyone under ~30, they won't know what you're talking about.

My existence proves this sentence wrong.

Re: An Efficient Way to Extract the Main Topics from a Sentence

#48

Post by Someone »

mindcrime wrote: post #42
shmageggy wrote: post #41

Earlier quoted context omitted.

SHRDLU is definitely amazing, especially given its age, but one's amazement is tempered a little bit (or maybe enhanced, depending on perspective) when you realize that it achieved what it did primarily through really great engineering rather that some fundamental insight about language. Since SHRDLU's world is so limited, Winograd was able to explicitly program every facet of its language understanding. Unsurprising…

Since SHRDLU's world is so limited, Winograd was able to explicitly program every facet of its language understanding. Unsurprisingly, this approach is totally not scalable and this reveals a little about why we don't have fully human-like language programs. That's a good point. It does lead one to wonder, however, if techniques inspired to SHRDLU could (or do) have application in domain-specific applications where t…

Firstly, we have done similar things. For example, we have/had http://en.wikipedia.org/wiki/METEO_System for weather reports (use "machine translation weather reports" to google Scientific literature. Among others, that finds information that work is being done on a Croatian version of this). I think there have been successes in the medical field, too, but cannot find them.

However, this 'knowledge engineering' approach to AI has fallen somewhat out of fashion a bit in favourite of statistical methods (however, I don't think anybody does statistics 'from scratch'. For example, in NLP, you could try to statistically learn the definite articles in English, but hard-coding that 'the' is the only one will get you results faster.

Re: An Efficient Way to Extract the Main Topics from a Sentence

#49

Post by syllogism »

microtonal wrote: post #45
syllogism wrote: post #33

Earlier quoted context omitted.

Yes, for English. There's a standard multi-lingual evaluation for statistical dependency parsers (the CoNLL 2007 data), but none for constituency parsers. I had a look at that paper, but didn't read it carefully. All I can really say is that there's a real evaluation problem between rule-based and statistical parsers. Rule-based parsers recover richer representations, but tend to have lower coverage over arbitrary da…

Rule-based parsers recover richer representations, but tend to have lower coverage over arbitrary data This parser usually has a coverage around ~95-97%. In one experiment we also parsed Flemish (which uses some constructions that wouldn't be considered grammatical in Dutch) and obtained a coverage of ~94%. It also has a robustness component that attempts to provide an analysis for as many constituents as possible if…

> Certainly. But you are not mentioning the other elephant in the room: Dutch is a free word order language and also permits very liberal ordering in the middle field. The rule-based grammar may benefit from the detailed constraints in the lexical attribute-value structures.

Mostly I didn't want to stick my neck out :p. It's easy to say something untrue about a language you don't know and haven't worked with.

I'd also be reluctant to assume which mechanisms were making a difference, because it's so hard to guess what cases are frequent and not easily inferred by a statistical model. One thing we can know is that the transition based parsers are best at producing projective dependency trees. The various techniques for non-projective shift-reduce parsing aren't very good.

Don't get me wrong --- I totally think the suggestion you offered makes sense, and it's a likely explanation. It's just that this stuff all very tricky.

Re: An Efficient Way to Extract the Main Topics from a Sentence

#50

Post by yolesaber »

run4yourlives wrote: post #32
teeja wrote: post #23

The sentence subject is one thing, the sentence topic might be quite another. Consider sentences like: "He joined the not-yet-famous Liverpool band in early 1958." To many human beings the topic is quickly obvious. Parsing won't do the trick.

The only reason that sentence is "obvious" to many people is because we have a reference to a famous band from Liverpool that got its start in the late 50's/early 60's that is already embedded in our brain's library of facts. Removed from that context human beings see that sentence as equally meaningless as a parser, because it is. I'd imagine many young people (who don't have the "correct" reference points) wouldn't…

This is why systems such as MIT's ConceptNet exist. NLTK and other NLP systems can provide the grammatical breakdown and ConceptNet can fill in the semantic / contextual gaps.
Post reply on HN