Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A computational linguistic farce in three acts

earningmyturns.org

Post reply on HN
Print view View original

41–50 of 56 posts

Re: A computational linguistic farce in three acts

#41

Post by dang »

aisofteng wrote: post #27
SomeStupidPoint wrote: post #25

Earlier quoted context omitted.

It's also a very strawman version of what I said, to the point of being wildly inaccurate.

Oh, you used the religiously verified word of "strawman"! What you said made little to no sense and had no backing. Yours was a perfect example of layman speculation without any basis. Nothing you said made any coherent sense, nor had any backing. They don't even deserve a response.

Your comments have repeatedly been violating the HN guidelines by being uncivil and/or unsubstantive and generally nasty. We ban accounts that do this, so please stop doing this, and instead post civilly and substantively (or not at all).

https://news.ycombinator.com/newsguidelines.html

https://news.ycombinator.com/newswelcome.html

Re: A computational linguistic farce in three acts

#42

Post by YeGoblynQueenne »

paulsutter wrote: post #32
YeGoblynQueenne wrote: post #26

Earlier quoted context omitted.

My reading is that Pereira doesn't think that deep learning has quite conquered language, and in this he's in complete disagreement both with Goldberg and Le Cunn's side (who both champion deep learning for NLP and claim that it has led to great advances in the field). For me the problem with NLP and deep learning, or indeeed any empirical method, is that the evaluation metrics we have are imperfect. Take BLEU scores…

Google translate is now based on a neural network and you can be sure they have solid metrics. By analogy Google search has a large panel of humans whose subjective feedback is used to test the quality of search algorithm variations.

>> you can be sure they have solid metrics.

Btw- no, I can't be sure of that. Why do you say I can? Do you know what metrics they use?

Re: A computational linguistic farce in three acts

#43

Post by unhammer »

YeGoblynQueenne wrote: post #26
paulsutter wrote: post #4

This relates to the big Twitter uproar over this blog post: https://medium.com/@yoav.goldberg/an-adversarial-review-of-a... And here's the meat of his response: > Idea! Let's go back to toy problems where we can create the test conditions easily, like the rationalists did back then (even if we don't realize we are imitating them). After all, Atari is not real life, but it still demonstrates remarkable RL progress. Le…

My reading is that Pereira doesn't think that deep learning has quite conquered language, and in this he's in complete disagreement both with Goldberg and Le Cunn's side (who both champion deep learning for NLP and claim that it has led to great advances in the field). For me the problem with NLP and deep learning, or indeeed any empirical method, is that the evaluation metrics we have are imperfect. Take BLEU scores…

Bit of an aside: Apparantly ChrF – character-level n-gram f-score – is the new hotness in evaluating MT systems http://www.aclweb.org/anthology/W/W15/W15-30.pdf#page=412

Re: A computational linguistic farce in three acts

#44

Post by YeGoblynQueenne »

SomeStupidPoint wrote: post #39
yorwba wrote: post #36

Earlier quoted context omitted.

My favorite example where you need semantics to get the syntax is "See the a are of I." +---+---+---+---+ | x | | | | I +---+---+---+---+ | | | | | K +---+---+---+---+ a b c d The map shows 8 ares, the a are of I is marked with a cross.

You guys seem confused: I'm not claiming that what you're calling semantics doesn't exist; I'm saying that it's merely convoluted syntactical rules, and calling it a different name is misleading. It would be correct to say that the sentence in isolation is a parse error, but with the diagram, it's merely elaborate syntax. My point isn't that there aren't higher order rules (and approximations) -- just that the divisi…

>> You guys seem confused: I'm not claiming that what you're calling semantics doesn't exist; I'm saying that it's merely convoluted syntactical rules, and calling it a different name is misleading.

Syntax and semantics, or structure and meaning, are completely different things- in fact they are entirely unrelated and their only association is by arbitrary convention (we all agree that certain structures are associated with specific meaning).

This is why you can do translation, for example- where you're essentially taking the semantics out of one kind of syntax and putting it into another.

In NLP it's easy enough to reproduce the structure of a corpus- simple, unsmoothed n-grams will do that well enough already and with a little more statistical elbow grease you can train a model that reproduces your text very well and even generates new text that looks quite resonable. Except of course that it rarely makes any sense at all. To generate text that is both syntactically correct and makes sense you need a lot more than that and we haven't really managed to do that except for very short durations (a few words at a time).

I'm saying: in NLP we can deal with structure very well indeed, but meaning is still a long way off. If it was just a matter of "more syntax", we'd have solved all our problems a long time ago.

Re: A computational linguistic farce in three acts

#45

Post by YeGoblynQueenne »

SomeStupidPoint wrote: post #24
hcs wrote: post #12

Earlier quoted context omitted.

What do you mean?

That we artificially decompose the process of accepting a sentence as, eg, proper English into two phases: syntactic correctness and semantic correctness. However, that distinction is arbitrary -- there is only the question of if the sentence is accepted by an agent (eg, person) as a well-formed sentence. Any full accounting of the class of well-formed sentences must embed the semantic concerns; violating semantics i…

>> violating semantics is a syntax error

That's a very strange thing to say. The thing with human language is you can say anything you like, including things that make no sense at all and things that are syntactically incorrect. You can easily find examples of meaningless, syntactically correct sentences, like Jabberwocky ("All mimsy were the Borogoves and the mome raths outgrabe" etc). It's also easy to find examples of sensible sentences with incorrect structure (see twitter.com).

In fact, what is "incorrect syntax" keeps changing all the time, but we can still say the same things as we always could (plus a probably infinite many new things besides). If syntax was tied to meaning as tight as you say, we'd probably have only one or two languages and no dialects. Language would be a static, unchanging thing and we'd need no NLP, or translators etc.

Re: A computational linguistic farce in three acts

#46

Post by YeGoblynQueenne »

SomeStupidPoint wrote: post #34
gizmo686 wrote: post #33

Earlier quoted context omitted.

1) Colorless green ideas sleep furiously. 2) Me gizmo. One of the above sentences is valid English, the other one is meaningful. That is the syntax/semantics distinction. The distinction is a bit fuzzy in places (for instance, inflectional morphology), but does exist.

I actually disagree with your assessment. You contend that the top sentence is valid English; I disagree with that. It subscribes to the "first order" rules (or an overly simplistic model), but isn't a sentence that an English speaker would use. Not being one that an English speaker would use makes it invalid English -- it's just a case where the first order approximation is wrong. Similarly, if your syntax rules rej…

>> It subscribes to the "first order" rules (or an overly simplistic model), but isn't a sentence that an English speaker would use.

Well, an English speaker did use it.

Btw, who do you consider an "English speaker"? Do I count as an English speaker? My native language is Greek but I speak English as a foreign language. I often say things that a native English speaker wouldn't say- but they convey a meaning that I wish to express. Do these utterances count as things that "an English speaker would use", or not?

I say they do. English speakers can say anything they like. In fact, they do, everyday, and as they do their language changes along with that.

Human language seems to be a lot more flexible than what you give it credit for. Semantics being just some sort of higher-order syntax (which btw we just haven't found yet) would make for a much more limited language ability than what we currently have. We'd be restricted to only a finite set of forms and we could only say a finite number of things. Obviously, that's not the case.

Re: A computational linguistic farce in three acts

#47

Post by YeGoblynQueenne »

unhammer wrote: post #43
YeGoblynQueenne wrote: post #26

Earlier quoted context omitted.

My reading is that Pereira doesn't think that deep learning has quite conquered language, and in this he's in complete disagreement both with Goldberg and Le Cunn's side (who both champion deep learning for NLP and claim that it has led to great advances in the field). For me the problem with NLP and deep learning, or indeeed any empirical method, is that the evaluation metrics we have are imperfect. Take BLEU scores…

Bit of an aside: Apparantly ChrF – character-level n-gram f-score – is the new hotness in evaluating MT systems http://www.aclweb.org/anthology/W/W15/W15-30.pdf#page=412

OK, but this still has the same problem as BLEU- it relies on comparisons to human scores, which are entirely subjective. I'm not saying they're not the best we got, but it's a big problem for machine translation that the only way to evaluate results is, essentially, comparing it to eyballing.

Re: A computational linguistic farce in three acts

#48

Post by ajuc »

YeGoblynQueenne wrote: post #40
paulsutter wrote: post #32

Earlier quoted context omitted.

Google translate is now based on a neural network and you can be sure they have solid metrics. By analogy Google search has a large panel of humans whose subjective feedback is used to test the quality of search algorithm variations.

This is something that needs to be repeated until everyone internalises it: for language pairs other than the "easy" ones Google translate sucks. I am Greek and translations from and to my language are utterly ridiculous, on the level of Bozo the clown doing the translation with his underpants on his head back to front. Typical example: I put in the Greek word for "swallow", the bird, and ask for the French translati…

I'm Polish. I google translate even from Slavic languages that are very close to Polish (Ukrainian, Slovak - it's like 50% understandable without translation) to English not to Polish, because X -> Polish google translation sucks.

Re: A computational linguistic farce in three acts

#50

Post by SomeStupidPoint »

YeGoblynQueenne wrote: post #46
SomeStupidPoint wrote: post #34

Earlier quoted context omitted.

I actually disagree with your assessment. You contend that the top sentence is valid English; I disagree with that. It subscribes to the "first order" rules (or an overly simplistic model), but isn't a sentence that an English speaker would use. Not being one that an English speaker would use makes it invalid English -- it's just a case where the first order approximation is wrong. Similarly, if your syntax rules rej…

>> It subscribes to the "first order" rules (or an overly simplistic model), but isn't a sentence that an English speaker would use. Well, an English speaker did use it. Btw, who do you consider an "English speaker"? Do I count as an English speaker? My native language is Greek but I speak English as a foreign language. I often say things that a native English speaker wouldn't say- but they convey a meaning that I wi…

There was an implied "in isolation" on that sentence -- what's proper with other clauses and sentences included isn't necessarily alone.

An English speaker used it as an example of a statement that would cause a parse error for most English speakers, and so it did. The speaker said it even caused such a reaction in them. I would argue that they weren't attempting to use English, but quasi-English in an attempt to communicate the boundaries of English to people who can parse English (which inherently has some ability to parse quasi-English).

I don't think it's useful to pretend "English" is a coherent class of parsing rules (either over time or over population) -- there's only a roughly similar set of parsers undergoing continuous memetic evolution, broken up into subsets that are more similar.

At the end of the day, English is as people who can parse some subset of it do -- and it might reach the point where it makes more sense to talk about English languages than an Enish language.

That being said, your last paragraph confuses me:

It's not obvious to me that we aren't restricted to a finite number of forms in language.

It's not clear to me why you think semantics being higher order syntax requires that it only be capable of finitely many forms.

(The rest of it seems dependent on those two conclusions.)

Post reply on HN