Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A Simple AI Capable of Basic Reading Comprehension

blog.ayoungprogrammer.com

Post reply on HN
Print view View original

41–50 of 64 posts

Re: A Simple AI Capable of Basic Reading Comprehension

#41

Post by tariqali34 »

kajecounterhack wrote: post #36
tariqali34 wrote: post #9

Earlier quoted context omitted.

>I think it is too much to call this "capable of basic reading comprehension". Surely, "simple sentence parser can answer reading comprehension questions" would be more correct? We test whether a human have "basic reading comprehension" by asking them reading comprehension questions. If they answer the questions successfully, then we assume that they have this "reading comprehension" skill. Therefore, if an AI can an…

We test human "reading comprehension" with the assumption that humans have an underlying world model. This is starkly different from a machine whose only ability is information retrieval -- this program (and more advanced versions e.g Freebase) parse and restructure text to be more retrievable. But there is no comprehension because there is no world model. You wouldn't call parsing a google search query "reading comp…

But if a computer is able to do the same thing as a human does (answer a simple question with pattern recognition and sentence parsing), then the internal mechanism of how a computer does it seems like mere trivia/jargon rather than anything fundamentally important to intelligence. A computer may have reading comprehension without having it in exactly the same way that a human does.

But it's true that we do have a underlying world model by which we understand passages. And that world model is pretty essential if we are to answer more complicated questions. So since it's pretty clear computers can handle the "informational retrieval" side of reading, we should now be focused on generating a "world model" based on text.

Re: A Simple AI Capable of Basic Reading Comprehension

#42

Post by blazespin »

wslh wrote: post #26
nyamhap wrote: post #14

Earlier quoted context omitted.

When was that xkcd posted? - couldn't not find a date. Since machine vision is such a fast moving field, I think date is relevant for understanding the context of this xkcd. Is image recognition of a bird so inconceivable at the moment? Perhaps the joke now would be - "I'll need one researcher and one year"

You are taking the xkcd strip very literally instead of understanding the central concept on it. You comment reminds me of this: http://www.uh.edu/engines/epi879.htm

I think the central concept is believe a smart engineer when he says something can be done, but not always when he says it can't be done.

Re: A Simple AI Capable of Basic Reading Comprehension

#43

Post by kajecounterhack »

tariqali34 wrote: post #41
kajecounterhack wrote: post #36

Earlier quoted context omitted.

We test human "reading comprehension" with the assumption that humans have an underlying world model. This is starkly different from a machine whose only ability is information retrieval -- this program (and more advanced versions e.g Freebase) parse and restructure text to be more retrievable. But there is no comprehension because there is no world model. You wouldn't call parsing a google search query "reading comp…

But if a computer is able to do the same thing as a human does (answer a simple question with pattern recognition and sentence parsing), then the internal mechanism of how a computer does it seems like mere trivia/jargon rather than anything fundamentally important to intelligence. A computer may have reading comprehension without having it in exactly the same way that a human does. But it's true that we do have a un…

  the internal mechanism of how a computer does it seems like 
  mere trivia/jargon rather than anything fundamentally 
  important to intelligence. A computer may have reading 
  comprehension without having it in exactly the same way that 
  a human does.
Yes, it is trivia what we decide to call it, and computers may someday have "intelligence" or "sentience" without having it in the same way that we do. However I think mechanically structuring data and building a querying tool over that cannot be considered "intelligence" or "comprehension" in the sense we seem to speak of here.

  since it's pretty clear computers can handle the 
  "informational retrieval" side of reading, we should now be   
  focused on generating a "world model" based on text.
Yes, that's what Google's working on with Knowledge Graph. It's quite good, it's integrated with search already.

Re: A Simple AI Capable of Basic Reading Comprehension

#44

Post by mrec »

aamar wrote: post #35
kajecounterhack wrote: post #22

The difference between this graph and propositional logic is only that the predicates joining concepts are arbitrary instead of logic operators. In that sense, this is like Google Knowledge Graph / Freebase. https://en.wikipedia.org/wiki/Propositional_calculus#Solvers Solving a set of propositional logic statements is NP-Complete. I'd argue "reading comprehension" is actually knowing the state of the world after a pi…

Classic example: The iron ball fell on the glass table, and it shattered. The glass ball fell on the iron table, and it shattered. Human readers will pick the correct antecedent for "it" in each case, so it's not ambiguous. But correct interpretation depends on knowing something about how likely glass and iron are to shatter. I love these kinds of examples, but I don't think they ought to significantly deter the kind…

These are called Winograd Schemas; you can read more about them and see more examples at https://www.cs.nyu.edu/davise/papers/WS.html

Re: A Simple AI Capable of Basic Reading Comprehension

#45

Post by vonnik »

This is a really neat, rules-based, Chomskyian NLP (as opposed to the statistical kind represented by Word2vec). It's an old division...

http://norvig.com/chomsky.html

The essential question is: Can it go from basic reading comprehension to advanced just by adding more rules. Is intelligence simply 10 million rules? If so, how do we go about creating new rules as language evolves? By hard-coding them, as in the example code?

The real test for general AI and NLP is how well it, well, generalizes; i.e. how well does it deal with situations we have not explicitly anticipated?

In my opinion, the fuzzy, statistical methods @davesullivan mentions have a better chance at generalizing (although they may well be augmented by rules-based AI).

If an AI doesn't have a good way of transferring what it knows to novel problems, then it is severely limited. It's treating the world like a canned problem with a finite number of possibilities, like chess or checkers, when in fact the world is much more complex.

The way DeepMind combines deep learning and reinforcement learning is one way of acknowledging that complexity.

Deep learning learns patterns in raw sensory data, which means it can ingest and handle the new. Reinforcement learning learns to perform actions over a series of unknown states, improving its choices by monitoring the rewards it receives for those actions. They both maximize within uncertainty, and I think that's our best bet going forward.

Because the world, and language, cannot be known in their entirety. The number, motion and interrelation of the atoms of air in the room where I'm typing this are all too large and complex to be computable. Their fluid dynamics can only be vaguely guessed at, not deterministically predicted in a few lines of code.

The trick will be to bridge the gap between the hard-coded, limited rules and the unlimited recombinations of language, which is inventing new rules and words all the time.

Re: A Simple AI Capable of Basic Reading Comprehension

#46

Post by aamar »

kajecounterhack wrote: post #40
aamar wrote: post #35

Earlier quoted context omitted.

Classic example: The iron ball fell on the glass table, and it shattered. The glass ball fell on the iron table, and it shattered. Human readers will pick the correct antecedent for "it" in each case, so it's not ambiguous. But correct interpretation depends on knowing something about how likely glass and iron are to shatter. I love these kinds of examples, but I don't think they ought to significantly deter the kind…

Yeah, it's a bit of a nitpick over definition of "comprehension" but the program itself is obviously useful, I mean it's a poor man's version of Freebase. Freebase/Google Knowledge Graph doesn't claim to "comprehend" anything, it is just a large graph datastructure with an efficient querying mechanism. That's what this is (...given this is a poor man's version, efficient querying mechanism may be lacking for larger g…

We don't have to worry about the philosophical questions of comprehension to find cases like this present challenges. These types of example (better articulated in mrec's link) are cases where we can imagine straightforward, plausible queries that the computer would be unable to answer: "what shattered?" or "what remains?" Some of the examples are such that simple web searches wouldn't be helpful in resolving the ambiguity; you'd need some "common knowledge." (Again: not that that's an insurmountable or necessarily relevant problem.)

Re: A Simple AI Capable of Basic Reading Comprehension

#47

Post by oh_sigh »

aamar wrote: post #35
kajecounterhack wrote: post #22

The difference between this graph and propositional logic is only that the predicates joining concepts are arbitrary instead of logic operators. In that sense, this is like Google Knowledge Graph / Freebase. https://en.wikipedia.org/wiki/Propositional_calculus#Solvers Solving a set of propositional logic statements is NP-Complete. I'd argue "reading comprehension" is actually knowing the state of the world after a pi…

Classic example: The iron ball fell on the glass table, and it shattered. The glass ball fell on the iron table, and it shattered. Human readers will pick the correct antecedent for "it" in each case, so it's not ambiguous. But correct interpretation depends on knowing something about how likely glass and iron are to shatter. I love these kinds of examples, but I don't think they ought to significantly deter the kind…

For those curious, this process is called "anaphora resolution", which is the process of understanding what a reference(usually a pronoun) actually refers to.

Re: A Simple AI Capable of Basic Reading Comprehension

#48

Post by reptation »

aamar wrote: post #35
kajecounterhack wrote: post #22

The difference between this graph and propositional logic is only that the predicates joining concepts are arbitrary instead of logic operators. In that sense, this is like Google Knowledge Graph / Freebase. https://en.wikipedia.org/wiki/Propositional_calculus#Solvers Solving a set of propositional logic statements is NP-Complete. I'd argue "reading comprehension" is actually knowing the state of the world after a pi…

Classic example: The iron ball fell on the glass table, and it shattered. The glass ball fell on the iron table, and it shattered. Human readers will pick the correct antecedent for "it" in each case, so it's not ambiguous. But correct interpretation depends on knowing something about how likely glass and iron are to shatter. I love these kinds of examples, but I don't think they ought to significantly deter the kind…

While this is a interesting point, grammarians will bristle at such an imprecise use of 'it', which should unambiguously refer to one antecedent.

Re: A Simple AI Capable of Basic Reading Comprehension

#49

Post by anigbrowl »

Nice work, but but isn't this way of breaking down sentence structure already a standard thing? I get the impression that the writer didn't find much on this in the AI literature, but reinvented a wheel that has been long-established in linguistics departments.

https://en.wikipedia.org/wiki/Sentence_diagram

I don't mean that in a dismissive way - even if it was a reinvention of the wheel it's still an elegant and useful one. It would be interesting to work up to larger chunks of text, and also to encapsulate ambiuities in some way, such that if presented with a sentence that admits of two meanings the program could honestly say 'I don't know, tell me more.'

Re: A Simple AI Capable of Basic Reading Comprehension

#50

Post by acd »

Very cool project. I would be good to have an AI similar to this and if it could read and comprehend lots of research articles. I dream of a AI capable of reading all research articles on the latest battery tech and then be able to understand and make recomendations from that. The question I would like us to ask the AI how would you create the worlds most efficient battery?
Post reply on HN