Finnish breaks natural language processors
twitter.com
Finnish breaks natural language processors
1–10 of 131 posts
Re: Finnish breaks natural language processors
#2Why can't the NLP split the compounds? I think this problem of splitting happens in multiple languages -- the submission mentions German, and then you have Chinese which doesn't have spaces, so you have to split words. (I understand that splitting Japanese into words is simpler.)
Re: Finnish breaks natural language processors
#3Finn here. There are countless memes about oddities of our language. The one I like the most is "kuusi palaa". It can mean;
the spruce is on fire
the spruce returns
the number six is on fire
the number six returns
six of them are on fire
six of them return
your moon is on fire
your moon returns
six pieces
Re: Finnish breaks natural language processors
#4What about Hungarian ?
Re: Finnish breaks natural language processors
#5Polish has some of those problems too, at least the ones with conjugations. The words mom (appearing in your contact list), mom (as in call mom) and mom (as in send money to mom) are different. That applies to proper names too, and it isn't regular. It's not easy to figure oujt who the user refers to if you just have the contacts list and the command.
Polish had problems way before nlp, for example "Annie has sent you a message" and "John has sent you a message" would be translated differently (because male/female), same for "2 minutes ago" and "5 minutes ago". Also programmatically formulating natural messages like "Dear John" is nigon impossible.
Re: Finnish breaks natural language processors
#6Finn here. There are countless memes about oddities of our language. The one I like the most is "kuusi palaa". It can mean; the spruce is on fire the spruce returns the number six is on fire the number six returns six of them are on fire six of them return your moon is on fire your moon returns six pieces Good luck for all implementing NLP :)
I wonder what oddities lie in Linus Torvalds' "perkeleen vittupää" (https://lkml.org/lkml/2013/7/13/132).
Re: Finnish breaks natural language processors
#7So, does that mean that if OpenAI's text generator turns out to be as "dangerous" as it's being marketed, we can all just switch to Finnish as lingua franca and Skynet won't have a chance? :)
Re: Finnish breaks natural language processors
#8Finn here. There are countless memes about oddities of our language. The one I like the most is "kuusi palaa". It can mean; the spruce is on fire the spruce returns the number six is on fire the number six returns six of them are on fire six of them return your moon is on fire your moon returns six pieces Good luck for all implementing NLP :)
I like the "dog example" too, which highlights the different forms/cases:
https://linustechtips.com/main/topic/72936-english-swedish-g...
I'm a recent immigrant to Finland, and while the regular grammar makes some things simple, the cases and suffixes are enough to drive me crazy! The simple things are easy to handle if you're reading a textbook, but remembering them in real-time when speaking is very hard.
Re: Finnish breaks natural language processors
#9There was a joke that Estonian is more closer to Japanese than any other European language. As to not say its related to Japanese, but how much it differs from European ones.
No future tense, no genders, but 14 cases.
This illustrates nicely where finno-ugric languages(Finnish, Hungarian, Estonian) reside compared to other European languages.
https://images.mentalfloss.com/sites/default/files/196.jpg
Benefit is that people are somewhat safe for scammers, as no translator does a good job. Downside is, none of the translators work and its way easier to just translate to english for example.
Re: Finnish breaks natural language processors
#10These are common problems for languages with complex morphology. Finnish, Japanese, Turkish and Hungarian are closely related in this sense and share a lot of NLP research.
While implementing spell checkers for these languages need a bit more effort than just compiling a list of words, it's far from an unsolved problem. AFAIK Zemberek[1] is a Turkish spellchecker that implements two-level finite state morphology.
Also, a small terminology fix: Verbs are "conjugated" but names/objects are "declinated".