11–20 of 45 posts
Are there IP considerations? Word2vec is patented.
System and method for generating a relationship network - K Franks, CA Myers, RM Podowski - US Patent 7,987,191, 2011 - http://www.google.com/patents/US7987191
As pre-built word vectors go, Conceptnet Numberbatch [1], introduced less flippantly as the ConceptNet Vector Ensemble [2], already outperforms this on all the measures evaluated in its paper: Rare Words, MEN-3000, and WordSim-353. This fact is hard to publicize because somehow the luminaries of the field decided that they didn't care about these evaluations anymore, back when RW performance was around 0.4. I have ha…
That said: LexVec gives quite good results on word-relatedness for using only distributional knowledge, and only from Wikipedia at that. Adding ConceptNet might give something that is more likely to be state-of-the-art.
I generally find that expert-derived ontologies suffer from bad coverage of low frequency items, rigidly discrete relationships, and are usually limited to a single language. That said, they're vastly better than nothing for a lot of tasks (same goes for WordNet).
Earlier quoted context omitted.
System and method for generating a relationship network - K Franks, CA Myers, RM Podowski - US Patent 7,987,191, 2011 - http://www.google.com/patents/US7987191
Would this really be usable in court? It seems super general to me, using a lot of common techniques. Silly question, is it infringement to use any part of the patent?
Here's a nice blog post about it: http://sebastianruder.com/word-embeddings-1/
It can process something like this: king - man + woman = queen
Neat-o.
Has anyone done any work on handing words that have overloading meanings? Something like 'lead' has two really distinct uses. It's really multiple words that happened to be spelt the same.
Earlier quoted context omitted.
That said: LexVec gives quite good results on word-relatedness for using only distributional knowledge, and only from Wikipedia at that. Adding ConceptNet might give something that is more likely to be state-of-the-art.
...And just distributional knowledge makes it easy to train new models on domain-specific corpora, or new languages. Is it possible to do the same with ConceptNet? I generally find that expert-derived ontologies suffer from bad coverage of low frequency items, rigidly discrete relationships, and are usually limited to a single language. That said, they're vastly better than nothing for a lot of tasks (same goes for W…
The whole idea of Numberbatch is that a combination of distributional and lexical knowledge is much better than either one alone.
BTW, ConceptNet is only partially expert-derived (much of it is crowd-sourced), aims not to be rigid like WordNet is, and is in a whole lot of languages.
"Retraining" ConceptNet itself is a bit of a chore, but you can do it. That is, you can get the source [1], add or remove sources of data, and rebuild it. Meanwhile, if you wanted to retrain word2vec's Google News skip-gram vectors, you would have to get a machine learning job at Google.
If anyone else is wondering what the heck "word embedding" means, it's a natural language processing technique. Here's a nice blog post about it: http://sebastianruder.com/word-embeddings-1/ It can process something like this: king - man + woman = queen Neat-o.
Has anyone done any work on handing words that have overloading meanings? Something like 'lead' has two really distinct uses. It's really multiple words that happened to be spelt the same.
My experience is that you can distinguish word senses, but it seems the data isn't good enough to improve anything but a task that specifically evaluates that same vocabulary of word senses.
I see a sibling comment with link to spaCy's sense2vec, which uses the coarsest possible senses -- one sense for nouns, one sense for verbs, one sense for proper nouns, etc. It's a start.
As pre-built word vectors go, Conceptnet Numberbatch [1], introduced less flippantly as the ConceptNet Vector Ensemble [2], already outperforms this on all the measures evaluated in its paper: Rare Words, MEN-3000, and WordSim-353. This fact is hard to publicize because somehow the luminaries of the field decided that they didn't care about these evaluations anymore, back when RW performance was around 0.4. I have ha…
Has anyone done any work on handing words that have overloading meanings? Something like 'lead' has two really distinct uses. It's really multiple words that happened to be spelt the same.
Well, there is Sense2Vec: https://github.com/spacy-io/sense2vec