Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Finnish breaks natural language processors

twitter.com

Post reply on HN
Print view View original

101–110 of 131 posts

Re: Finnish breaks natural language processors

#101

Post by foldr »

gizmo686 wrote: post #94
foldr wrote: post #89

Earlier quoted context omitted.

>Regardless, under either analysis "will" would have just as much claim to being a tense as "run" has to being a verb. If you're punning on "tense" to refer to a lexical category, then sure. But we're talking about tense in the sense of past, present and future. In many textbook analyses, lots of things go in T. For example, "can" and "might" go in T. It doesn't follow that, say, "I might go to the store" and "I can…

I would say that "I can/might go to the store" go to the store having different tenses is what we mean when we say "can" and "might" are lexixally T. Even restricting ourselves to the syntactic features, I have still never seen an arguement for or against English having a syntactic future tense feature that did not come down to UG, or a disagreement about which model is simpler, and neither side is convoluted enough…

The two sentences have different words occupying the "T" head, but they don't have different tenses. What you're saying is just a pun on "tense".

Re: Finnish breaks natural language processors

#102

Post by apendleton »

simonh wrote: post #80
gorbachev wrote: post #58

Earlier quoted context omitted.

No, there is no variation on pronunciation. They sound exactly the same when spoken. The only way to infer what's meant is from context.

My favourite homophone example is a 92 character poem in Chinese[0], where every syllable is a slight variation on Shi. [0] https://en.wikipedia.org/wiki/Lion-Eating_Poet_in_the_Stone_... Interestingly when I showed the actual text to my wife, who is Chinese, she didn't understand what was funny about it until I asked her to read it to me, when she burst out laughing. That's when I found out she doesn't sub-vocalise,…

These aren't really homophones though, right? Because they differ in tone?

Re: Finnish breaks natural language processors

#103

Post by yorwba »

apendleton wrote: post #102
simonh wrote: post #80

Earlier quoted context omitted.

My favourite homophone example is a 92 character poem in Chinese[0], where every syllable is a slight variation on Shi. [0] https://en.wikipedia.org/wiki/Lion-Eating_Poet_in_the_Stone_... Interestingly when I showed the actual text to my wife, who is Chinese, she didn't understand what was funny about it until I asked her to read it to me, when she burst out laughing. That's when I found out she doesn't sub-vocalise,…

These aren't really homophones though, right? Because they differ in tone?

There are only 4 different tones, so by the pigeonhole principle there must be lots of homophones.

On the other hand, most of the characters used to be pronounced differently when they were still commonly used as individual words, so the poem only works because it mixes modern pronunciation with classical grammar.

Re: Finnish breaks natural language processors

#104

Post by DFHippie »

foldr wrote: post #81
DFHippie wrote: post #77

Earlier quoted context omitted.

> There is no unit of English syntax which (i) behaves like a verb and (ii) has a distinct future form. This seems like an arbitrary and peculiar way to define tense specifically to exclude periphrastic languages like English. If these are the rules by which we are arguing, the argument doesn't seem that interesting. By this argument colloquial French and German have no past tense because they use periphrastic constr…

It is indeed the case that (spoken) French and German have no past tense, strictly speaking. (I don't speak German, so I don't know to exactly what extent the German imperfect is unused in everyday speech. I think it may not be as unused as the passé simple in French.) Tense is a piece of technical terminology with a particular definition. There's nothing very peculiar about the definition that I can see. The definit…

It's not a question of taking linguists' word for it. It's a question of taking particular linguists' word for it. The discussion in that language log is interesting. You will notice that it isn't simply people calling "amen".

Certain things are settled: "will" in English behaves like other English auxiliary verb (setting aside those cases where it doesn't, as in contractions). Saying "English has no future tense", though, is settled only within the confines of a particular linguistic model with a particular technical definition of the term "tense", of which there are many.

Re: Finnish breaks natural language processors

#105

Post by kaitai »

frosted-flakes wrote: post #97

It's interesting that Nokia's auto-completion worked well for Finnish, but modern solutions don't work. > Joose Rajamäki 🇫🇮🇪🇺 @joose_rajamaeki Feb 20 > Yes, autocompletion [on phones] exists. But I hardly ever manage to compose a message where it wouldn't encounter new words. Also, it doesn't know the inflections, so it needs to encounter each word in each possible form before suggesting them. > Joose Rajamäki 🇫…

On their Windows phones Nokia's Finnish autocomplete was actually even a bit better -- it suggested suffixes and was pretty accurate. I wonder how they trained it.

Re: Finnish breaks natural language processors

#106

Post by Jenz »

iainmerrick wrote: post #39
a_imho wrote: post #36

Earlier quoted context omitted.

What is the benefit of genders anyway? Especially when gender neutral alternatives are available. It never made much sense to me, they introduce a lot of complexity for very little. Worse, nowadays you have all the problems of misaddressing and offending folks.

There was an interesting/amusing post here the other day about learning German and how it relates to programming concepts. They suggested that gendered words work as a kind of parity bit for error correction.

> There was an interesting/amusing post here the other day about learning German and how it relates to programming concepts.

This one: https://news.ycombinator.com/item?id=19109371

Re: Finnish breaks natural language processors

#107

Post by bravura »

wolfgke wrote: post #57
iainmerrick wrote: post #39

Earlier quoted context omitted.

There was an interesting/amusing post here the other day about learning German and how it relates to programming concepts. They suggested that gendered words work as a kind of parity bit for error correction.

> There was an interesting/amusing post here the other day about learning German and how it relates to programming concepts. They suggested that gendered words work as a kind of parity bit for error correction. Slightly off topic (not about grammatical genus, but about German's case system): I often compare German's case system to a type system in a programming language: the verb expects the sentence's object in the…

I am trying to learn German and gender is infuriating there. Why? Because you can't simply scan an existing sentence and color-code the words based upon simple heuristics to guess the gender.

The first thing you learn is that three genders are indicated in the nominative case using der (masculine) / die (feminine) / das (neuter).

So you see "der" in a sentence and that thing must be masculine, right? Nope, because "der" is feminine in the dative case.

The gender system in German wouldn't be such garbage if the articles weren't overloaded. Imagine if in your type system "float" could mean two different types, and you needed more information to figure out which type it is (like, for example, memorizing the type of the object its referring to). It breaks the idea of locality.

Re: Finnish breaks natural language processors

#108

Post by DFHippie »

peterwwillis wrote: post #100
a_imho wrote: post #36

Earlier quoted context omitted.

What is the benefit of genders anyway? Especially when gender neutral alternatives are available. It never made much sense to me, they introduce a lot of complexity for very little. Worse, nowadays you have all the problems of misaddressing and offending folks.

"Tommy, Frank, and Susan burned their homework and fed it to the dog." "Who fed it to the dog?" "That person." Overhearing this conversation, you don't know who they were pointing at. If they said "She did." , you know who they were pointing at. There are many other situations where gender provides extra data, which is convenient. Our languages keep having gender, so it's clearly been advantageous from an evolutionar…

Noun classes are advantageous. Gender happens to be a useful way to categorize nouns because mostly we like to talk about other people. Gender (and animacy -- "it" versus "s/he") split things up efficiently for many conversations, where, say, paper versus non-paper wouldn't.

Re: Finnish breaks natural language processors

#109

Post by ajuc »

babuskov wrote: post #96
ajuc wrote: post #35

Earlier quoted context omitted.

Doesn't solve the problem with "X out of Y".

It does. You have two numbers, so you have to split it into X and Y part. When I look at your example, 1 plik is always 1 plik regardless of the Y quantity and 100 always go with "ze" regardless of X quantity. It's a little bit tricky to set up, but it works. "Copied X of Y files" needs to split into two translation units: "Copied X" and "of Y files".

That's "not supported" in my book. How should English or Chinese developer know to split this? What about translating to other languages with different splits?

There's no system to handle this, so each special case must be embedded in the software itself, instead of properly doing it in internationalization package.

So - in effect - nobody even tries.

Re: Finnish breaks natural language processors

#110

Post by appsonify »

hpaavola wrote: post #3

Finn here. There are countless memes about oddities of our language. The one I like the most is "kuusi palaa". It can mean; the spruce is on fire the spruce returns the number six is on fire the number six returns six of them are on fire six of them return your moon is on fire your moon returns six pieces Good luck for all implementing NLP :)

Or counting in Finnish :D

https://www.youtube.com/watch?v=AuNs1v_jtfs

Post reply on HN