Hey! I am Sanjaya, the lead developer of wink-lemmatizer, and the rest of the wink family. We are trying to break down NLP and ML into atomic building blocks. I am happy to answer any questions and gather feedback about wink located at https://github.com/winkjs :)
Wink-lemmatizer
21–23 of 23 posts
Re: Wink-lemmatizer
#22Ok, so how does it work? Which stemming algorithm? It's not helpful unless we know what it's doing under the covers.
The lemmatizer is an adaptation of https://wordnet.princeton.edu/documentation/morphy7wn. It conjugates the verb and adjectives to their base form, whereas the plural nouns are converted to singular form. The wink-pos-tagger (https://github.com/winkjs/wink-pos-tagger) leverages the lemmatizer to automatically find lemma of each word as per its part of speech.
For stemming, we have wink-porter2-stemmer (https://github.com/winkjs/wink-porter2-stemmer) that uses Porter Stemmer Algorithm V2 by Dr Martin F Porter.
Re: Wink-lemmatizer
#23any plans for multi-language support?
As of now wink-tokenizer (https://github.com/winkjs/wink-tokenizer) supports multiple scripts, therefore, it can also tokenize sentences in languages like Hindi, Marathi, French, German etc. We are working on extending multi-lingual support to other components including this lemmatizer.