Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Finnish breaks natural language processors

twitter.com

Post reply on HN
Print view View original

81–90 of 131 posts

Re: Finnish breaks natural language processors

#81

Post by foldr »

DFHippie wrote: post #77
foldr wrote: post #63

Earlier quoted context omitted.

No, it isn't just a matter of orthography. English has no future tense on any reasonable analysis. There is no unit of English syntax which (i) behaves like a verb and (ii) has a distinct future form. This is a completely uncontroversial point. See e.g. http://languagelog.ldc.upenn.edu/nll/?p=897 for further discussion. > If we define words as the unit that carries a single beat in speaking Note that this definition…

> There is no unit of English syntax which (i) behaves like a verb and (ii) has a distinct future form. This seems like an arbitrary and peculiar way to define tense specifically to exclude periphrastic languages like English. If these are the rules by which we are arguing, the argument doesn't seem that interesting. By this argument colloquial French and German have no past tense because they use periphrastic constr…

It is indeed the case that (spoken) French and German have no past tense, strictly speaking. (I don't speak German, so I don't know to exactly what extent the German imperfect is unused in everyday speech. I think it may not be as unused as the passé simple in French.)

Tense is a piece of technical terminology with a particular definition. There's nothing very peculiar about the definition that I can see. The definition denies English a future tense because English doesn't have periphrastic units which (i) behave like verbs and (ii) have a distinct future form.

For whatever reason, people are really resistant to taking linguists' word for it on what technical terms within linguistics mean. You don't find people on here trying to argue with physicists about whether they should maybe adopt a different definition of "force" or "mass" or "electron".

Re: Finnish breaks natural language processors

#82

Post by gizmo686 »

foldr wrote: post #76
gizmo686 wrote: post #74

Earlier quoted context omitted.

There is a standard analysis of English in generative grammar where verbs and tense are are seperate nodes at deep structure [0], and we get inflected verbs through some form of V-T movement [0]. Under this analysis, English clearly has a future tense. It just gets pronounced as its own word (will) instead instead of being involved in V-T movement. A more popular analysis is to replace "T" with "I", and do everything…

>There is a standard analysis of English in generative grammar where verbs and tense are are seperate nodes at deep structure Yes, there is. >Under this analysis, English clearly has a future tense. No, this does not follow. You can just as easily (and more correctly) analyze "will" as a modal auxiliary like "can" or "would". Supposing it's true that "will" is the daughter of I/T/whatever, it doesn't follow that the…

I just have an undergrad degree in linguistics.

I am looking at this question from the perspective of x-bar theory, and defining "tense" as as a feature of the syntactic catagory that populates the populates the "T" node at deep structure (or a feature of the I catagory). [0].

As I alluded to in my comment, I have never seen a convincing arguement in either direction for wheather "will" is a T or a V; and this was an active discussion in my syntax class about 3 years ago where the proffesor (a syntaxtician) had also said that she never saw a compelling arguement either way. If you know of one, please sharer.

Edit: I think you edited your post after I responded. I would argue that all languages have the same featureset (including tense) at d structure, but that is going far afield.

Edit2: there is also a notion of tense at the sementic level. And English seems to offer a gramaticalized way of providing the sementic future tense, regardless of how you look at the corresponding syntax/morphology.

If we accept that "will" is a syntactic "T" and carries a sementic meaning of "future" then how is it unreasonable to call it a future tense? Sure, there are debatable parts to the above assumptions, but it is a stretch to say that no analysis has future tense. Heck, there is a universal grammar analysis that would say English (and Chinese) has all the tenses, and the only question is which ones surface as distinct pronounciations.

[0] This leads to a notationally weird case where "to" is a tense with a -tense feature.

Re: Finnish breaks natural language processors

#83

Post by gio2j »

Morphology of Finnish language has of course been studied, and spell checkers for it exist.

E.g. open-source: https://voikko.puimula.org/ (and its online version https://oikofix.com/contact?lang=en, where you can also play with the analysis/parsing capabilities)

Of course, the more complicated morphology makes it less straightforward than English where simple word list based approaches are sufficient.

Re: Finnish breaks natural language processors

#84

Post by foldr »

gizmo686 wrote: post #82
foldr wrote: post #76

Earlier quoted context omitted.

>There is a standard analysis of English in generative grammar where verbs and tense are are seperate nodes at deep structure Yes, there is. >Under this analysis, English clearly has a future tense. No, this does not follow. You can just as easily (and more correctly) analyze "will" as a modal auxiliary like "can" or "would". Supposing it's true that "will" is the daughter of I/T/whatever, it doesn't follow that the…

I just have an undergrad degree in linguistics. I am looking at this question from the perspective of x-bar theory, and defining "tense" as as a feature of the syntactic catagory that populates the populates the "T" node at deep structure (or a feature of the I catagory). [0]. As I alluded to in my comment, I have never seen a convincing arguement in either direction for wheather "will" is a T or a V; and this was an…

Putting "will" in T doesn't mean that T necessarily has to bear a future tense feature. It could e.g. have a present tense feature.

Re: Finnish breaks natural language processors

#85

Post by gizmo686 »

foldr wrote: post #84
gizmo686 wrote: post #82

Earlier quoted context omitted.

I just have an undergrad degree in linguistics. I am looking at this question from the perspective of x-bar theory, and defining "tense" as as a feature of the syntactic catagory that populates the populates the "T" node at deep structure (or a feature of the I catagory). [0]. As I alluded to in my comment, I have never seen a convincing arguement in either direction for wheather "will" is a T or a V; and this was an…

Putting "will" in T doesn't mean that T necessarily has to bear a future tense feature. It could e.g. have a present tense feature.

Are you getting at the distinction between "will" being the T and "will" being the daughter of T? Regardless, under either analysis "will" would have just as much claim to being a tense as "run" has to being a verb. You can still argue over wheather there is a syntactic feature of future tense (which is a different question) which I have never seen a solid arguement for either way.

Re: Finnish breaks natural language processors

#86

Post by vorg »

foldr wrote: post #66
vorg wrote: post #65

Earlier quoted context omitted.

> There is no unit of English syntax which [...] Here you're defining syntax around your orthographic definition of what a word is. > "Is he really, in your opinion, to see her tomorrow?" The opinion words "really" and "in your opinion" added into that syntactic structure parallel the manner that opinion word "bloody" is added into a lexical structure to become "abso-bloody-lutely".

>Here you're defining syntax around your orthographic definition of what a word is. No, I'm not. As I said, there is no unit in English (regardless of whether or not it is written as a single word) that both behaves like a verb and has a distinct future form. >The opinion words "really" and "in your opinion" added into that syntactic structure parallel the manner that opinion word "bloody" is added into a lexical str…

> "is-to-see" is not a contiguous unit

You can insert stuff between "is" and "to-see" (and between "to" and "see", I hope you agree) because English mostly parses into left-branching trees, which is the only reason why you can't insert stuff between "see" and "-ing". Natural languages don't liberally mix left-branching structures and right-branching ones because they're hard for people to understand. E.g. Pinker's "The rapidity that the motion that the wing that the hummingbird has has has is remarkable" is grammatically correct but not easily understandable. The contiguity or lack of it operates on a different conceptual plane to whether a "word" can be inflected or not.

> loose analogy

To take some example language from your reply, some people speak this as two words, others (e.g. some Americans) as a compound word. Because the concept of a word is not well defined, the boundary between morphology and syntax is blurred. Putting "is" and "to" noncontiguously before a verb is the same as putting "-ed" after a verb. They are both "inflections".

Re: Finnish breaks natural language processors

#87

Post by foldr »

vorg wrote: post #86
foldr wrote: post #66

Earlier quoted context omitted.

>Here you're defining syntax around your orthographic definition of what a word is. No, I'm not. As I said, there is no unit in English (regardless of whether or not it is written as a single word) that both behaves like a verb and has a distinct future form. >The opinion words "really" and "in your opinion" added into that syntactic structure parallel the manner that opinion word "bloody" is added into a lexical str…

> "is-to-see" is not a contiguous unit You can insert stuff between "is" and "to-see" (and between "to" and "see", I hope you agree) because English mostly parses into left-branching trees, which is the only reason why you can't insert stuff between "see" and "-ing". Natural languages don't liberally mix left-branching structures and right-branching ones because they're hard for people to understand. E.g. Pinker's "T…

Look, if you have a totally radical reanalysis of English syntax according to which "is...to" can function as some kind of discontinuous inflectional affix, I wish you luck developing that.

It's clearly thousands of miles away from being a generally accepted analysis of English syntax. So unless you've written up your analysis in detail somewhere, there's not much further to discuss.

Re: Finnish breaks natural language processors

#89

Post by foldr »

gizmo686 wrote: post #85
foldr wrote: post #84

Earlier quoted context omitted.

Putting "will" in T doesn't mean that T necessarily has to bear a future tense feature. It could e.g. have a present tense feature.

Are you getting at the distinction between "will" being the T and "will" being the daughter of T? Regardless, under either analysis "will" would have just as much claim to being a tense as "run" has to being a verb. You can still argue over wheather there is a syntactic feature of future tense (which is a different question) which I have never seen a solid arguement for either way.

>Regardless, under either analysis "will" would have just as much claim to being a tense as "run" has to being a verb.

If you're punning on "tense" to refer to a lexical category, then sure. But we're talking about tense in the sense of past, present and future. In many textbook analyses, lots of things go in T. For example, "can" and "might" go in T. It doesn't follow that, say, "I might go to the store" and "I can go to the store" have a different tense.

Re: Finnish breaks natural language processors

#90

Post by plq »

Lasokki wrote: post #48
stonewhite wrote: post #27

Turkish being a member of ural-altaic language family, just like Finnish, suffers from similar problems. Google Translate improved much over the years yet it still generates laughable text at best. Stemmers used to create show-stopper word stems (don't really know the current situation). Although, it imported a lot of technical terms from various european languages, making technical texts seem to be more legible due…

Ural-Altaic language family is today considered an obsolete concept [1] and the families are considered unrelated [1] https://en.wikipedia.org/wiki/Ural%E2%80%93Altaic_languages

The models / software created for one language mostly fit others. Eg. Two-level finite state morphology was invented for Finnish and was very successfully adopted to parse Turkish words.

So the opinions of Linguists aside, the languages that make up the ural-altaic family are not that far apart from each other.

Post reply on HN