Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Finnish breaks natural language processors

twitter.com

Post reply on HN
Print view View original

51–60 of 131 posts

Re: Finnish breaks natural language processors

#51

Post by xg15 »

a_imho wrote: post #36
DocG wrote: post #9

There was a joke that Estonian is more closer to Japanese than any other European language. As to not say its related to Japanese, but how much it differs from European ones. No future tense, no genders, but 14 cases. This illustrates nicely where finno-ugric languages(Finnish, Hungarian, Estonian) reside compared to other European languages. https://images.mentalfloss.com/sites/default/files/196.jpg Benefit is that…

What is the benefit of genders anyway? Especially when gender neutral alternatives are available. It never made much sense to me, they introduce a lot of complexity for very little. Worse, nowadays you have all the problems of misaddressing and offending folks.

A pragmatic advantage is that you use different pronouns to refer to multiple different persons (or things, with grammatical gender) in the same sentence without ambiguity. In programming terms, you have a "local variable" slot for each pronoun, instead of just a single one with gender neutrality.

Re: Finnish breaks natural language processors

#52

Post by uasm »

hpaavola wrote: post #3

Finn here. There are countless memes about oddities of our language. The one I like the most is "kuusi palaa". It can mean; the spruce is on fire the spruce returns the number six is on fire the number six returns six of them are on fire six of them return your moon is on fire your moon returns six pieces Good luck for all implementing NLP :)

> "kuusi palaa"

Can we get a bit more detail here? Are those variations on pronunciation? How does a native Finn able to infer the correct meaning?

Re: Finnish breaks natural language processors

#53

Post by isani »

saagarjha wrote: post #50
isani wrote: post #40

Earlier quoted context omitted.

Japanese is usually written without spaces. Words and sentences just run into each other. When writing in hiragana (syllabic characters), word boundaries are often ambiguous. Englishwouldbemuchhardertoparseifwrittenlikethis.

I have no stake in natural language processing, but it looks to me like a computer might be able to do a pretty good job at splitting that given a dictionary.

Sure, you can get pretty far with a fairly simple solution. But lot of the time, you get two (or more) ways to split the string into dictionary words. For a simple English example, is it "justice was served" or "just ice was served"?

Re: Finnish breaks natural language processors

#55

Post by saagarjha »

isani wrote: post #53
saagarjha wrote: post #50

Earlier quoted context omitted.

I have no stake in natural language processing, but it looks to me like a computer might be able to do a pretty good job at splitting that given a dictionary.

Sure, you can get pretty far with a fairly simple solution. But lot of the time, you get two (or more) ways to split the string into dictionary words. For a simple English example, is it "justice was served" or "just ice was served"?

I guess that’s where context will have to be considered. Those two are valid sentences, so presumably humans are using context to distinguish between them, right?

Re: Finnish breaks natural language processors

#56

Post by szatkus »

isani wrote: post #40
szatkus wrote: post #19

Earlier quoted context omitted.

What's the problem with Japanese? It's highly regular language, so it should be easy to tokenize. AFAIR the whole language has maybe few irregular verbs, compared to few hundreds in English.

Japanese is usually written without spaces. Words and sentences just run into each other. When writing in hiragana (syllabic characters), word boundaries are often ambiguous. Englishwouldbemuchhardertoparseifwrittenlikethis.

Isn't it a case when parsing other languages from speech? Are there any audible cues between words when we speak?

Re: Finnish breaks natural language processors

#57

Post by wolfgke »

iainmerrick wrote: post #39
a_imho wrote: post #36

Earlier quoted context omitted.

What is the benefit of genders anyway? Especially when gender neutral alternatives are available. It never made much sense to me, they introduce a lot of complexity for very little. Worse, nowadays you have all the problems of misaddressing and offending folks.

There was an interesting/amusing post here the other day about learning German and how it relates to programming concepts. They suggested that gendered words work as a kind of parity bit for error correction.

> There was an interesting/amusing post here the other day about learning German and how it relates to programming concepts. They suggested that gendered words work as a kind of parity bit for error correction.

Slightly off topic (not about grammatical genus, but about German's case system): I often compare German's case system to a type system in a programming language: the verb expects the sentence's object in the suitable case. If you use a wrong type, it gives a compile error. In most cases, the case (type) that you have to use makes sense, but there are rare cases where you simply have to learn the case. The latter is like having to use a (programming) library that was designed by some other programmer with an interface that you would design differently.

Just like you have to write a computer program that gives no compile error (e.g. because of typing), in German, you also have to formulate your sentences in a way that gives no compile error to its case (type) system.

Re: Finnish breaks natural language processors

#58

Post by gorbachev »

uasm wrote: post #52
hpaavola wrote: post #3

Finn here. There are countless memes about oddities of our language. The one I like the most is "kuusi palaa". It can mean; the spruce is on fire the spruce returns the number six is on fire the number six returns six of them are on fire six of them return your moon is on fire your moon returns six pieces Good luck for all implementing NLP :)

> "kuusi palaa" Can we get a bit more detail here? Are those variations on pronunciation? How does a native Finn able to infer the correct meaning?

No, there is no variation on pronunciation. They sound exactly the same when spoken.

The only way to infer what's meant is from context.

Re: Finnish breaks natural language processors

#59

Post by ivoras »

a_imho wrote: post #46
nyagaga wrote: post #42

Earlier quoted context omitted.

I don't know what you mean exactly, but genders can be something intrinsic to a language and I don't think they have been introduced as such. If you wanted to call a table in Italian, you could choose the masculine or feminine noun for it (tavolo - tavola), no neuter form though. Ask the table first.

Why would it be intrinsic? Somewhere in their evolution they must have been introduced for some reason I would guess. Also, they could be removed the same way if there is no benefit distinguishing between a masculine and feminine table.

Could it mean that gender was something so obvious and useful as a concept that early people, while trying to figure out the world around them and how it all fits together, just naturally extended the idea to just about everything?

It doesn't really make sense to anthropomorphise things once you're an adult, but I bet everyone can remember trying to figure out the world as a kid and just assuming things are of similar nature as people.

I think that even languages which currently don't have gender forms of words, like English, still have the concept psychologically firmly embedded in the culture. The generic "cat" is more female than male. The "tom-cat" is only male. A "rock", I'll bet, is also subconsciously male, and a "river" more female than male. It's all there, even if not explicit in language. OTOH, it's not really universal, different languages assign gender pretty much randomly - but they do assign it.

Social determinism and radical neutrality go against the grain of this "making sense of the world" phase of development, and probably don't result in a practical world-view.

Re: Finnish breaks natural language processors

#60

Post by vorg »

foldr wrote: post #45
DocG wrote: post #9

There was a joke that Estonian is more closer to Japanese than any other European language. As to not say its related to Japanese, but how much it differs from European ones. No future tense, no genders, but 14 cases. This illustrates nicely where finno-ugric languages(Finnish, Hungarian, Estonian) reside compared to other European languages. https://images.mentalfloss.com/sites/default/files/196.jpg Benefit is that…

>No future tense Many European languages also lack a future tense. English, for example. ( https://www.quora.com/Is-it-true-that-English-has-no-future-... )

> English has no inflected future tense like Latin and Romance languages. In linguistics, “tense” refers generally to how a verb is inflected

English has no inflected future tense because of the way words are usually defined. Verbs in English only have 4 types of inflections because words are defined as whatever is surrounded by spaces in writing. If we define words as the unit that carries a single beat in speaking, so "He is-to-see-it tomorrow" is 3 words, then verbs also have prefixed inflections. English then does have an inflected future tense.

The "is to" future form isn't mentioned in discussions on future tense very often but seems to have the most predetermination, whereas "is going to" and "will" indicate more volition by the doer.

Post reply on HN