Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How would you use an accurate Natural Language Understanding (NLU) API?

maluu.ba

Post reply on HN
Print view View original

1–10 of 45 posts

Re: How would you use an accurate Natural Language Understanding (NLU) API?

#3

Post by apetresc »

rhizome wrote: post #2

How accurate? Accuracy is a value, not a state.

Our technology was initially designed to be used with external ASR. For sentences within our 18 domains, where the ASR is correct, we see accuracy (defined by mapping into the current domain and action) around 90%. The reality of course though is ASR is sometimes wrong, which does decrease our accuracy in practice.

The best direct demonstration of practical accuracy is just try out the app ;)

Re: How would you use an accurate Natural Language Understanding (NLU) API?

#4

Post by hooande »

I would use a good natural language understanding api to do basic research, ideally monitoring many different feeds for topics of interest. It could also be a good way to monitor trends or competitors.

I can't think of many uses cases for natural language as an interface to a traditional website. If there's something that's more convenient to access with text/speech, the user can just use set commands.

On the topic of building a personal assistant app, from what I've seen the real value lies in the act of interactive scheduling. You can't say "schedule a meeting with george" without checking with george's schedule to make sure he's available, and potentially re-scheduling the event at a time of mutual availability. Of course this goes far beyond the scope of understanding language, but I think this is a big part of what will differentiate a good personal assistant app from Siri.

EDIT: Someone on the thread brought up customer service, which is also a great opportunity. Interacting with a company's service representatives is a huge time sink for people everywhere, especially on the phone. Anything that lets people spend less time on hold is a huge win for mankind.

Re: How would you use an accurate Natural Language Understanding (NLU) API?

#5

Post by vhf »

I cannot answer your question -- I have no idea. (edited : well, why not for "customer service", along with the traditional FAQ ? Or use it instead of the useless "solve this problem" useless multiple choice walkthrough in Windows or such ? )

But I upvoted because I really liked to discover this company, what it does, and more importantly how it does it. Thanks for the clear examples and output provided !

Re: How would you use an accurate Natural Language Understanding (NLU) API?

#8

Post by drewch »

hooande wrote: post #4

I would use a good natural language understanding api to do basic research, ideally monitoring many different feeds for topics of interest. It could also be a good way to monitor trends or competitors. I can't think of many uses cases for natural language as an interface to a traditional website. If there's something that's more convenient to access with text/speech, the user can just use set commands. On the topic o…

That's a pretty sweet idea. I wonder how difficult it is to set up sharing of calendars, and the permissions that are involved in it.

Re: How would you use an accurate Natural Language Understanding (NLU) API?

#9

Post by languagehacker »

From what I read, I don't really see a robust NLU engine here. I see a probabilistic transformation engine (maybe bootstrapped with some linguistic data structures) that identifies 7 buckets and then rephrases the query or request into a format better suited to achieve the desired result in that engine.

This isn't too far from saying that you have a search engine that "knows" how to answer questions, because you have hard-coded it to omit question words and transform question phrases into something declarative in hopes of finding the best match (e.g. "What is the largest mountain in Asia?" --> "The largest mountain in Asia", and then list the top search result from Google).

There's this tremendous amount of pragmatism that goes into some of the example questions, too. If I ask someone if they can recommend a good sushi place, personally, I'm probably more interested in the best sushi hit they've ever had than whether then place I can walk to from work is decent. You're going to have to make a lot of assumptions that you cannot adequately contextualize to take a broad question like that and say "Well, here are two within two blocks of you." I might actually be ranking proximity lower than, say, quality, popularity, or memorability. By hard-coding such assumptions into your reasoning, you're not in the trade of understanding natural language; you're in the trade of doing exactly what Bing does now, at a smaller scale, with a few more buckets.

If you're really into building NLU, I have a few somewhat snarky suggestions for you: 1) Swing by a cryogenics facility and check back in with us in 20-30 years. Repeat until cognitive scientists have figured out a way to transform human thought into something that can be quantified and transformed into words.

2) Do some reading in cognitive science to understand the immensely difficult problem it is to quantify meaning, both within the context of a discourse as well as discourse-agnostic (archetypal meaning versus contextualized meaning). Look at some charts, and realize that word meaning is not only relative to other words in the same discourse, but other words in shared classes.

3) Read up on Grice, and understand that, despite his philosophical approach to linguistics, his points on implicature and the value of what goes unsaid in a conversation lend as much to context as what goes said. Then try to sit back and figure out how to program and analyze what's going unsaid in a search query.

What I'm trying to say is that I would do nothing with this API because I can't trust it to deliver on its very lofty promises. Natural language understanding is essentially a pipe dream in the current state of computer science and bioinformatics. Word meaning is so idiosyncratic that it is a miracle that humans can generally agree on what a word represents; most of these are extremely fuzzy, and cannot be divorced from the contexts in which they are uttered. It's far too difficult to contextualize even a sentence outside of a pure information retrieval standpoint.

I appreciate what you're aiming at here, but I doubt you'll ever be able to succeed at it. That being said, if you rephrased it into something like a decision engine, I probably wouldn't be so irritated by it, and might even give it a shot.

Post reply on HN