Live data from Hacker News

A Simple AI Capable of Basic Reading Comprehension

blog.ayoungprogrammer.com

51–60 of 64 posts

Re: A Simple AI Capable of Basic Reading Comprehension

#51
Thanks for sharing your work. As a hobby, I have spent years working on extracting semantic ontologies from natural language, so it was fun to see someone else's take on the problem.

As other have mentioned, you will make progress more efficiently if you survey the linguistics literature, where a tremendous number of very smart people have spent decades grappling with the same essential problems.

Heterodox linguistics is a veritable goldmine of ideas that can be implemented in AI. My favorite approach is Richard Hudson's "word grammar," which you can read about here: http://www.phon.ucl.ac.uk/home/dick/wg.htm

Word grammar is particularly well suited for coding, because it strips away lots of arbitrary linguistic formalisms in favor of a flexible, network-centric framework. Some of the core principles, like default inheritance, were actually taken directly from computer science.

Re: A Simple AI Capable of Basic Reading Comprehension

#52
post #45

This is a really neat, rules-based, Chomskyian NLP (as opposed to the statistical kind represented by Word2vec). It's an old division... http://norvig.com/chomsky.html The essential question is: Can it go from basic reading comprehension to advanced just by adding more rules. Is intelligence simply 10 million rules? If so, how do we go about creating new rules as language evolves? By hard-coding them, as in the examp…

> The essential question is: Can it go from basic reading comprehension to advanced just by adding more rules. Is intelligence simply 10 million rules? If so, how do we go about creating new rules as language evolves? By hard-coding them, as in the example code?

I believe that it can go from basic reading comprehension to more advanced by adding many rules but of course manually adding them is not very feasible or scaleable.

> In my opinion, the fuzzy, statistical methods @davesullivan mentions have a better chance at generalizing (although they may well be augmented by rules-based AI).

I agree that a statistical model would be better since it will be able to handle more complexity. It would be much easier to train the rules from a dataset instead of hard coding all of them and it would be able to adapt to new rules as well. However, I could not find a good data set for the task I wanted.

Re: A Simple AI Capable of Basic Reading Comprehension

#53

Earlier quoted context omitted.

> I was intrigued by the initial transcript, but disappointed that it was edited (lightly). For the question "Why did Mary cheer?" the screenshot showed that the simple AI literally answered "because IT be HER FIRST TIME WINNING". This is correct, but the author edited it into a correct sentence for us. For some reason, the library I used to get the present tense of "was" is "be". I had a manually fix for this but ac…

Thanks for the reply! Thanks for sharing your work with us as well. It's impressive. It's not disappointing, it's quite a feat. I suppose I would say that what I was trying to say is that "parsing", while impressive, is not a large part of "understanding" in my opinion. To use an analogy: almost by definition, compilers parse languages like C++ much better than humans do (basically perfectly, unless there's literally…

> So my basic objection is that you're really calling this reading comprehension, but I don't think anything is actually being "understood"; just parsed. A better title would be as I suggested: simple AI correctly answers reading comprehension questions.

I would argue that my program can understand the relationship between different objects but I agree that it does not understand the meaning of the relationships.

> I couldn't answer a REAL reading-comprehension test about this: I just have no idea what it's REALLY talking about, I don't actually understand it. (Obviously on a syntactic level, it's not hard to parse.) I don't know what over-expression is, I don't know what a kinase is, I don't know what phosphorylation is. I don't understand the text. But if the questions are simple, perhaps I could answer some reading comprehension questions about this by parroting back quotations from it. Syntactically, there's nothing difficult here. I just don't understand it.

I would also argue that you are doing very basic reading comprehension here. You might not know the meaning of individual objects, but you understand that "what is wrong" is "the overlapping substrate specificities of many AGC kinases, it is likely that the over-expression of one member of this kinase subfamily will result in the phosphorylation of substrates that are normally phosphorylated by another AGC kinase". You might not know what that whole phrase means, but you understand that its related to "whats wrong" with "studying the physiological role of AGC kinases by overexpressing the active forms in cells". I agree that my program is unable to do full comprehension in not understanding the meaning of objects and relationships, but it can do very basic comprehension in understanding what the relationships are.

> So I think it's unfair to call sentence parsing real reading comprehension, even if sometimes reading comprehension tests fail to differentiate between the two. You can parse sentences perfectly while understanding nothing.

I did not really call it "real" reading comprehension, but "basic" reading comprehension. But I suppose "basic reading comprehension" is still a little of a stretch. I think the real question here is: how can you really determine if a program can understand something? What does understanding something really mean? It is difficult to define something like this and it seems we need some kind of "Turing test" for understanding.

> I did find your work very interesting, thank you.

Thanks!

Re: A Simple AI Capable of Basic Reading Comprehension

#54
post #26

Earlier quoted context omitted.

You are taking the xkcd strip very literally instead of understanding the central concept on it. You comment reminds me of this: http://www.uh.edu/engines/epi879.htm

I think the central concept is believe a smart engineer when he says something can be done, but not always when he says it can't be done.

I think that it means that thing #1 and thing $2 might appear to be just as difficult to untrained eyes while one would require very little work because of what's already available (libraries, data or whatever) and the other would be a very large task because most of the work hasn't been done yet (but of course a different engineer might be able to find more efficient ways to do things)

Re: A Simple AI Capable of Basic Reading Comprehension

#55

Some thoughts about creating a system like this: Any successful implementation of comprehension must progressively enhance the world model based on additional information. Furthermore it must understand some basic rules, such as, "Any subject, set of subjects, or actions can be represented multiple ways." So if you said, "Mary's brother is Sam," or "Mary has a brother named Sam," or "Mary's brother is named Sam," the…

> Further, if you mention that John also has a brother named Sam, and then you mention Sam in an ambiguous context, the program should be smart enough to ask, "Which Sam? Mary's brother or John's?" Option 3: The only Sam, since Mary and John are brothers :)

Back to the drawing board, you've failed this Turing test! (Mary is no one's brother.)

Re: A Simple AI Capable of Basic Reading Comprehension

#57
post #35

Earlier quoted context omitted.

Classic example: The iron ball fell on the glass table, and it shattered. The glass ball fell on the iron table, and it shattered. Human readers will pick the correct antecedent for "it" in each case, so it's not ambiguous. But correct interpretation depends on knowing something about how likely glass and iron are to shatter. I love these kinds of examples, but I don't think they ought to significantly deter the kind…

While this is a interesting point, grammarians will bristle at such an imprecise use of 'it', which should unambiguously refer to one antecedent.

Who are these 'grammarians', and why should anyone care what makes them bristle? They evidently have a complete lack of appreciation for the semantic content of sentences and can only see syntax. What a very poor view of language that would be.

Professional writers, editors, and teachers of language would be perfectly fine with those sentences, because they understand language as being made up of syntax AND semantics.

Re: A Simple AI Capable of Basic Reading Comprehension

#58
this post and many of the comments here don't necessarily have a "semiotic" or "usage-based language acquisition" or "emergent-grammar" nature about them ...

which is fine ... just different results ...

specifically:

1. meaning is usage. 2. structure emerges from usage.

this post and many of the comments have a world view akin to:

1. meaning is structure. 2. usage emerges from structure.

what i mean by this is that the "analysis" of the text doesn't exist in the same "world" as the text.

meaning that its nothing like "real" (natural) language.

what i mean further by this is simple:

humans don't "use" natural language... it is an emergent property of other systems of externalized behaviors and such by individual humans.

and such emergent properties and systems are also evident in any development of this actual system and many of the comments here.

producing the comprehension introduces in-comprehensible things ... or at best just divorces systems (discontinuous) ... at which point any thing can be any thing, so debating it as such here doesn't even matter (value) .

im not exactly sure what im saying here but it is akin to:

1. nthorder cybernetics (mostly like 3rd and 4th and such) 2. autopoesis (humberto maturana and francisco valera)

im going through the same type of analysis regarding "activity stream" type APIs which use an "actor verb object" type form ... (usage from structure) ...

Re: A Simple AI Capable of Basic Reading Comprehension

#60

The difference between this graph and propositional logic is only that the predicates joining concepts are arbitrary instead of logic operators. In that sense, this is like Google Knowledge Graph / Freebase. https://en.wikipedia.org/wiki/Propositional_calculus#Solvers Solving a set of propositional logic statements is NP-Complete. I'd argue "reading comprehension" is actually knowing the state of the world after a pi…

there is a linguistic research called "usage based" and also "emergent grammar" that i find interesting regarding subjects like this:

1. meaning is usage. 2. structure emerges from usage.

meaning we don't use grammar to produce language, it is an emergent property (irreducible) of "pre-linguistic pragmatics" ...

Post reply on HN