Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Finnish breaks natural language processors

twitter.com

Post reply on HN
Print view View original

61–70 of 131 posts

Re: Finnish breaks natural language processors

#61

Post by Agentlien »

aasasd wrote: post #25

I still haven't figured out how Germans use swipe keyboards on smartphones with the monster compound sentence-words. If they do.

There's the same problem in Swedish. I usually swipe the words separately or, if they're short, I type them manually and then go back to swiping. The decision to type them usually comes when I've tried a few times to swipe it before realising it's just not in the dictionary.

It's actually really frustrating. I absolutely love swiping, but it doesn't feel like it works very well with Swedish.

Re: Finnish breaks natural language processors

#62

Post by josmala »

Whatitat90 wrote: post #6
hpaavola wrote: post #3

Finn here. There are countless memes about oddities of our language. The one I like the most is "kuusi palaa". It can mean; the spruce is on fire the spruce returns the number six is on fire the number six returns six of them are on fire six of them return your moon is on fire your moon returns six pieces Good luck for all implementing NLP :)

I wonder what oddities lie in Linus Torvalds' "perkeleen vittupää" ( https://lkml.org/lkml/2013/7/13/132 ).

Its literal translation is.

Satan's cunt-head

Re: Finnish breaks natural language processors

#63

Post by foldr »

vorg wrote: post #60
foldr wrote: post #45

Earlier quoted context omitted.

>No future tense Many European languages also lack a future tense. English, for example. ( https://www.quora.com/Is-it-true-that-English-has-no-future-... )

> English has no inflected future tense like Latin and Romance languages. In linguistics, “tense” refers generally to how a verb is inflected English has no inflected future tense because of the way words are usually defined. Verbs in English only have 4 types of inflections because words are defined as whatever is surrounded by spaces in writing. If we define words as the unit that carries a single beat in speaking,…

No, it isn't just a matter of orthography. English has no future tense on any reasonable analysis. There is no unit of English syntax which (i) behaves like a verb and (ii) has a distinct future form. This is a completely uncontroversial point. See e.g. http://languagelog.ldc.upenn.edu/nll/?p=897 for further discussion.

> If we define words as the unit that carries a single beat in speaking

Note that this definition clearly won't do what you want it to do (even ignoring the imprecision of "single beat"), as the units you want to treat as verbs can be separated to an arbitrary degree. E.g. "Is he really, in your opinion, to see her tomorrow?"

Re: Finnish breaks natural language processors

#64

Post by emilfihlman »

uasm wrote: post #52
hpaavola wrote: post #3

Finn here. There are countless memes about oddities of our language. The one I like the most is "kuusi palaa". It can mean; the spruce is on fire the spruce returns the number six is on fire the number six returns six of them are on fire six of them return your moon is on fire your moon returns six pieces Good luck for all implementing NLP :)

> "kuusi palaa" Can we get a bit more detail here? Are those variations on pronunciation? How does a native Finn able to infer the correct meaning?

Finnish is pronounced extremely regularly. The way it's written is the way it's pronounced (in almost all cases).

Re: Finnish breaks natural language processors

#65

Post by vorg »

foldr wrote: post #63
vorg wrote: post #60

Earlier quoted context omitted.

> English has no inflected future tense like Latin and Romance languages. In linguistics, “tense” refers generally to how a verb is inflected English has no inflected future tense because of the way words are usually defined. Verbs in English only have 4 types of inflections because words are defined as whatever is surrounded by spaces in writing. If we define words as the unit that carries a single beat in speaking,…

No, it isn't just a matter of orthography. English has no future tense on any reasonable analysis. There is no unit of English syntax which (i) behaves like a verb and (ii) has a distinct future form. This is a completely uncontroversial point. See e.g. http://languagelog.ldc.upenn.edu/nll/?p=897 for further discussion. > If we define words as the unit that carries a single beat in speaking Note that this definition…

> There is no unit of English syntax which [...]

Here you're defining syntax around your orthographic definition of what a word is.

> "Is he really, in your opinion, to see her tomorrow?"

The opinion words "really" and "in your opinion" added into that syntactic structure parallel the manner that opinion word "bloody" is added into a lexical structure to become "abso-bloody-lutely".

Re: Finnish breaks natural language processors

#66

Post by foldr »

vorg wrote: post #65
foldr wrote: post #63

Earlier quoted context omitted.

No, it isn't just a matter of orthography. English has no future tense on any reasonable analysis. There is no unit of English syntax which (i) behaves like a verb and (ii) has a distinct future form. This is a completely uncontroversial point. See e.g. http://languagelog.ldc.upenn.edu/nll/?p=897 for further discussion. > If we define words as the unit that carries a single beat in speaking Note that this definition…

> There is no unit of English syntax which [...] Here you're defining syntax around your orthographic definition of what a word is. > "Is he really, in your opinion, to see her tomorrow?" The opinion words "really" and "in your opinion" added into that syntactic structure parallel the manner that opinion word "bloody" is added into a lexical structure to become "abso-bloody-lutely".

>Here you're defining syntax around your orthographic definition of what a word is.

No, I'm not. As I said, there is no unit in English (regardless of whether or not it is written as a single word) that both behaves like a verb and has a distinct future form.

>The opinion words "really" and "in your opinion" added into that syntactic structure parallel the manner that opinion word "bloody" is added into a lexical structure to become "abso-bloody-lutely".

You seem to have missed that fact that even if you remove "really" and "in your opinion", the unit you identified as a verb is still broken up in the example I gave. In "Is he to see her tomorrow?", "is-to-see" is not a contiguous unit. Although the idea of treating the subject pronoun as a verbal infix might be charming (English as Mohawk!), the subject could be a much larger phrase, as in e.g. "Is the man in the red hat to see her tomorrow?"

It is simply not true that adjunction of adverbials has the same properties as expletive infixation. Even if there is some loose analogy between these two processes, I fail to see how it's relevant to units defined in terms of a single "beat in speaking".

Re: Finnish breaks natural language processors

#67

Post by raverbashing »

a_imho wrote: post #46
nyagaga wrote: post #42

Earlier quoted context omitted.

I don't know what you mean exactly, but genders can be something intrinsic to a language and I don't think they have been introduced as such. If you wanted to call a table in Italian, you could choose the masculine or feminine noun for it (tavolo - tavola), no neuter form though. Ask the table first.

Why would it be intrinsic? Somewhere in their evolution they must have been introduced for some reason I would guess. Also, they could be removed the same way if there is no benefit distinguishing between a masculine and feminine table.

I believe the intrinsicality of it has to do with alliterations/rhymes/rhythm (more in the vowel than in the consonantal sounds)

Similar to effects of (past) umlauts in English or the a/an difference.

Re: Finnish breaks natural language processors

#68

Post by MereInterest »

saagarjha wrote: post #55
isani wrote: post #53

Earlier quoted context omitted.

Sure, you can get pretty far with a fairly simple solution. But lot of the time, you get two (or more) ways to split the string into dictionary words. For a simple English example, is it "justice was served" or "just ice was served"?

I guess that’s where context will have to be considered. Those two are valid sentences, so presumably humans are using context to distinguish between them, right?

The murderer came to my dinner party, and I had it all planned. In one of the ice cubes, I had frozen arsenic. The murderer would eat the same food, drink the same drink, and nobody would guess that they would die on leaving. When the evening was over, I knew what I would tell people.

Justicehadbeenserved.

Re: Finnish breaks natural language processors

#69

Post by usrusr »

xg15 wrote: post #51
a_imho wrote: post #36

Earlier quoted context omitted.

What is the benefit of genders anyway? Especially when gender neutral alternatives are available. It never made much sense to me, they introduce a lot of complexity for very little. Worse, nowadays you have all the problems of misaddressing and offending folks.

A pragmatic advantage is that you use different pronouns to refer to multiple different persons (or things, with grammatical gender) in the same sentence without ambiguity. In programming terms, you have a "local variable" slot for each pronoun, instead of just a single one with gender neutrality.

Like having separate registers for even and for odd numbers. Now try building a good compiler for that... (no bitshift trickery allowed)

Re: Finnish breaks natural language processors

#70

Post by arka2147483647 »

hpaavola wrote: post #3

Finn here. There are countless memes about oddities of our language. The one I like the most is "kuusi palaa". It can mean; the spruce is on fire the spruce returns the number six is on fire the number six returns six of them are on fire six of them return your moon is on fire your moon returns six pieces Good luck for all implementing NLP :)

This is a specificially constructed corner case, not a general finnish thing:

- kuu = moon

- kuusi = spruce

- palaa = returns

- pala = piece

From those you can see how one can intrepent the compound word (with or without suffix) however you wish.

Eg:

- kuu+si = your moon (where 'si' is the suffix which means 'your')

- pala+a = pieces (where 'a' suffix is the plural)

Post reply on HN