Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Wikipedia2Vec – A tool for learning embeddings of words and entities

wikipedia2vec.github.io

Post reply on HN
Print view View original

11–17 of 17 posts

Re: Show HN: Wikipedia2Vec – A tool for learning embeddings of words and entities

#11

Post by gitgud »

Off Topic:

If you click this post link, then click "Features" on the website, the browser 'back' button breaks. Clicking back multiple times doesn't get you back to Hacker News.

Can anyone explain what's going on here? Seems pretty annoying.

Re: Show HN: Wikipedia2Vec – A tool for learning embeddings of words and entities

#12

Post by ikuyamada »

riku_iki wrote: post #6

Curious if there are any benchmarks against Fasttext, Elmo, etc..

We did not add Fasttext to our benchmarks because of a minor technical issue but we will work on it. Further, to conduct a fair comparison with ELMo, I think it is needed to use extrinsic tasks such as question answering and textual entailment.

Re: Show HN: Wikipedia2Vec – A tool for learning embeddings of words and entities

#13

Post by ikuyamada »

rpedela wrote: post #2

This is awesome! Is there a plan to support all the major embedding algorithms?

Regarding word embedding algorithm, I am interested in supporting other models that uses subword information (e.g., Fasttext). Further, there have been proposed various recent models to learn entity representations from KB, and I plan to work on them.

Re: Show HN: Wikipedia2Vec – A tool for learning embeddings of words and entities

#14

Post by alexcnwy »

Very cool!

I'm currently busy with my MSc thesis on learning to link plain text documents to semantically relevant Wikipedia articles (and some of the cool machine learning-y things you can do from there).

I have 2 questions about your work:

1. I'm not sure if you're familiar with Doc2Vec but it allows you to train a Word2Vec model while also learning a vector for each document in the training corpus. Wikipedia is commonly used as a training corpus so you can get a "DocVec" for each Wikipedia article in the same vector space as your Word2Vec model (i.e. the DocVec for the Wikipedia page "Machine Learning" is nearby the WordVec for "mathematics"). Did you consider/compare with using Doc2Vec to learn a vector for each Wikipedia page and then use those as your entity vectors?

2. Your "Features" page says you convert an entity name to a link pointing to an entity if the entity name is unambiguou". In the case of ambiguous entities from a link (which happens often - my research is only learning links to Wikipedia articles from plain text documents), did you consider using the entity vectors (or some simple model built on top of the word vectors of the target page) to disambiguate?

Re: Show HN: Wikipedia2Vec – A tool for learning embeddings of words and entities

#15

Post by ikuyamada »

alexcnwy wrote: post #14

Very cool! I'm currently busy with my MSc thesis on learning to link plain text documents to semantically relevant Wikipedia articles (and some of the cool machine learning-y things you can do from there). I have 2 questions about your work: 1. I'm not sure if you're familiar with Doc2Vec but it allows you to train a Word2Vec model while also learning a vector for each document in the training corpus. Wikipedia is co…

Thanks :) 1) I think learning entity embeddings using the Doc2Vec (paragraph vector) model is an interesting idea, but we did not test it. 2) This tool was initially developed to address the entity linking task. Mapping words and entities into a same vector space enables to model the contextual information that is useful for entity linking. For details, please refer to this paper: Joint Learning of the Embedding of Words and Entities for Named Entity Disambiguation: https://arxiv.org/pdf/1601.01343.pdf

Re: Show HN: Wikipedia2Vec – A tool for learning embeddings of words and entities

#16

Post by riku_iki »

ikuyamada wrote: post #12
riku_iki wrote: post #6

Curious if there are any benchmarks against Fasttext, Elmo, etc..

We did not add Fasttext to our benchmarks because of a minor technical issue but we will work on it. Further, to conduct a fair comparison with ELMo, I think it is needed to use extrinsic tasks such as question answering and textual entailment.

You can build very simple 100 lines benchmark e.g. doing IMDB dataset classification, and use ELMO and your embeddings, and see who will perform better..

Re: Show HN: Wikipedia2Vec – A tool for learning embeddings of words and entities

#17

Post by ikuyamada »

riku_iki wrote: post #16
ikuyamada wrote: post #12

Earlier quoted context omitted.

We did not add Fasttext to our benchmarks because of a minor technical issue but we will work on it. Further, to conduct a fair comparison with ELMo, I think it is needed to use extrinsic tasks such as question answering and textual entailment.

You can build very simple 100 lines benchmark e.g. doing IMDB dataset classification, and use ELMO and your embeddings, and see who will perform better..

Thank you for your feedback! I am also interested in conducting experiments on extrinsic tasks such as text classification. In addition to word embeddings, Wikipedia2Vec also contains entity embeddings which are likely beneficial for these tasks, so I would like to design a model that uses both the word embeddings and entity embeddings.
Post reply on HN