Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A computational linguistic farce in three acts

earningmyturns.org

Post reply on HN
Print view View original

11–20 of 56 posts

Re: A computational linguistic farce in three acts

#11

Post by naturalgradient »

paulsutter wrote: post #4

This relates to the big Twitter uproar over this blog post: https://medium.com/@yoav.goldberg/an-adversarial-review-of-a... And here's the meat of his response: > Idea! Let's go back to toy problems where we can create the test conditions easily, like the rationalists did back then (even if we don't realize we are imitating them). After all, Atari is not real life, but it still demonstrates remarkable RL progress. Le…

Leading somewhat off-topic, but this has also sparked a rather frank debate on r/machinelearning about some of the things discussed in the review, in particular arxiv flag painting:

https://www.reddit.com/r/MachineLearning/comments/6gke6a/d_r...

Re: A computational linguistic farce in three acts

#12

Post by hcs »

SomeStupidPoint wrote: post #7
JohnStrange wrote: post #3

I kind of disagree with some of the premises in the article. I've seen an HPSG for German in the late 90s that was able to parse almost any sentence I could throw at it correctly from a syntactic perspective. The main problem for natural language understanding is not parsing and not even the semantic and pragmatic representations per se, it has always been the understanding. This requires an adequate knowledge repres…

Semantics is syntax. All semantics has ever been about is not causing parse errors during the decoding step of the sentence, and the constraints imposed on that. 'Syntax' is usually confined to "low level" concerns, while 'semantics' to those above, but the distinction is arbitrary and artificial. There is no meaning but usage.

What do you mean?

Re: A computational linguistic farce in three acts

#13

Post by vedant »

throwawuyar3231 wrote: post #8

I have to wonder if English is really the best language for NLP research. Things like the Winograd schemas which have attracted a lot of attention simply aren't possibilities in other languages. Why not start working with more structured agglutinative * languages like Japanese/Korean and Indic family (Sanskrit esp.) . How about other European languages ? Are they better structured empirically ? I hear German is very…

One reason is that the amount of training data is many many orders of magnitude smaller.

FWIW it seems the structure you're talking about exploiting is at a morphological and syntactic level, which modern language models tend to effectively handle. Semantics are a much harder problem.

Re: A computational linguistic farce in three acts

#14

Post by [deleted] »

jesuslop wrote: post #5
JohnStrange wrote: post #3

I kind of disagree with some of the premises in the article. I've seen an HPSG for German in the late 90s that was able to parse almost any sentence I could throw at it correctly from a syntactic perspective. The main problem for natural language understanding is not parsing and not even the semantic and pragmatic representations per se, it has always been the understanding. This requires an adequate knowledge repres…

Recently Coecke comments on Gärdenfors geometric meaning in the context of his categorical semantics that I'm finding interesting, in arXiv:1608.01402. What I would welcome is a computational link relating that semantics and oldie semantic-network based ideas. For instance in arXiv:1706.00526 description logic based knowledge representation is cast in string diagrammatic, categorical terms, and that at least puts the…

[deleted]

Re: A computational linguistic farce in three acts

#16

Post by throwaway-1209 »

The whole field of NLP and computational linguistics reminds me of that joke where a drunk is looking for his keys under a street lamp instead of where he actually lost them.

This is true in particular of anything that pertains to reasoning and knowledge representation. People still are trying to "infer rules" and do logical, rather than probabilistic reasoning. I get why that is. To me though, the kind of real life reasoning that humans do seems heavily probabilistic and contextual, Bayesian almost. And there's next to no notable work going on in that direction.

Re: A computational linguistic farce in three acts

#17

Post by johnbender »

throwaway-1209 wrote: post #16

The whole field of NLP and computational linguistics reminds me of that joke where a drunk is looking for his keys under a street lamp instead of where he actually lost them. This is true in particular of anything that pertains to reasoning and knowledge representation. People still are trying to "infer rules" and do logical, rather than probabilistic reasoning. I get why that is. To me though, the kind of real life…

I don't think these two things are mutually exclusive.

As far as I'm aware there is work underway to take logical constructions and integrate them with probablistic machine learning to do things like force zero probabilities in impossible input cases. That is encoding domain knowledge into the model directly in the form of symbolic reasoning.

I mean even Bayesian nets require some encoding of causality​ right? Maybe I'm reading to much of "blah symbolic reasoning is worthless" in your comment?

Re: A computational linguistic farce in three acts

#18

Post by lgessler »

throwawuyar3231 wrote: post #8

I have to wonder if English is really the best language for NLP research. Things like the Winograd schemas which have attracted a lot of attention simply aren't possibilities in other languages. Why not start working with more structured agglutinative * languages like Japanese/Korean and Indic family (Sanskrit esp.) . How about other European languages ? Are they better structured empirically ? I hear German is very…

> I hear German is very grammatical, and that Hungarian is ... erm odd ?

Just want to point out that "grammatical" probably isn't the word you want here. Every language is grammatical by definition in the sense that there are rules that govern its sound system, word formation system, syntax, etc.

The concept you're getting at, though--that some languages are easier for computer programs and/or speakers of Indo-European languages to understand--is sound.

Re: A computational linguistic farce in three acts

#19

Post by aisofteng »

hcs wrote: post #12
SomeStupidPoint wrote: post #7

Earlier quoted context omitted.

Semantics is syntax. All semantics has ever been about is not causing parse errors during the decoding step of the sentence, and the constraints imposed on that. 'Syntax' is usually confined to "low level" concerns, while 'semantics' to those above, but the distinction is arbitrary and artificial. There is no meaning but usage.

What do you mean?

It doesn't matter what he meant, all that matters is how he said it; your question is meaningless, for after all there is no meaning but usage.

(If it isn't clear, this comment is snide to GP.)

Re: A computational linguistic farce in three acts

#20

Post by YeGoblynQueenne »

So, I understand this blog post is about something else completely (the internet argument started by Yoav Goldberg on Medium, reportedly) but for me the really interesting part is the historical information in it. I wish Fernando Pereira could find the time to expound a bit on all those parenthetical notes in his blog post, perhaps even write a short book on the history of AI.

AI is kind of a strange beast like that: it's gone through a few very different phases and it's difficult for one person to understand all of them equally well. Which of course makes it even harder to avoid reinventing wheels and repeating mistakes. A bit of history would do us all a world of good.

Btw, I'm getting the feeling most people here will probably hear of Fernando Pereira for the fist time but he has a very long career in AI and NLP. He was a prominent symbolicist, with some important contributions to logic programming (he was one of the co-founders of Quintus, the company that sold the first commercial Prolog, along with Warren, Byrd and others). Then he turned to statistical AI and now he's a VP at Google (a.k.a. the den of the connectionists, if I may be so bold). He's probably one of the few computer scientists around who understands both symbolic and statistical AI in equal measures. If anyone is qualified to talk about their relative merits, that's him.

(and if I sound like a bit of a fangirl- that is because I basically am. Pereira is one of my logic programming heroes and a great teacher to me, albeit unbeknownst to him :)

Post reply on HN