Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LexVec, a word embedding model written in Go that outperforms word2vec

github.com

Post reply on HN
Print view View original

41–45 of 45 posts

Re: LexVec, a word embedding model written in Go that outperforms word2vec

#41

Post by oelmekki »

aweinstock wrote: post #31
oelmekki wrote: post #24

Earlier quoted context omitted.

Would love such kind of intro too, especially since I'm working on a product that could greatly benefit from NPL and neural networks. Could a kind person provide a good reference for something we could learn from? Or (as I fear), are we past this short time in the beginning of a technic/science (I think about computing, here) when you can learn without going through academic studies?

This is fairly readable (high-level) post on word embeddings: http://colah.github.io/posts/2014-07-NLP-RNNs-Representation...

Great read, thanks!

I know understand what word vectors are and how they are useful (which, btw, allows me to make way more sense of the output of the program in this HN post :D ), plus more generalities about the field.

Anyone in the same position than me (wanting to learn the basics) should read this.

Re: LexVec, a word embedding model written in Go that outperforms word2vec

#42

Post by oelmekki »

iraphael wrote: post #38
aweinstock wrote: post #31

Earlier quoted context omitted.

This is fairly readable (high-level) post on word embeddings: http://colah.github.io/posts/2014-07-NLP-RNNs-Representation...

And this is one that explains everything OP needs to understand the confusing sentence from their intro: http://mccormickml.com/2016/04/19/word2vec-tutorial-the-skip...

Thanks :) I feel like some key concepts are a bit beyond my reach, but that they are such a small quantity that it can be a good starting point for googling around. Thank for your help!

Re: LexVec, a word embedding model written in Go that outperforms word2vec

#43

Post by oelmekki »

knicholes wrote: post #30
oelmekki wrote: post #24

Earlier quoted context omitted.

Would love such kind of intro too, especially since I'm working on a product that could greatly benefit from NPL and neural networks. Could a kind person provide a good reference for something we could learn from? Or (as I fear), are we past this short time in the beginning of a technic/science (I think about computing, here) when you can learn without going through academic studies?

While dense and not necessarily for the mathematically faint of heart, I've learned a ton about NLP and ANNs through http://cs224d.stanford.edu/ .

It's quite an investment of time, but this is a good answer to "do we have to do academic studies to learn this stuff?"

=> Maybe, but nowadays you don't have to quit your job and move close to an university to learn, you can just have online courses.

Thanks :)

Re: LexVec, a word embedding model written in Go that outperforms word2vec

#44

Post by SergeyHack »

rpedela wrote: post #27

Slightly off-topic, but I thought this would be a good place to ask. Are there any word embedding tools which take a Lucene/Solr/ES index as input and output a synonyms file which can be used to improve search recall?

Not quite about creating synonyms, but in the same area there is Semantic Vectors https://github.com/semanticvectors/semanticvectors.

They process Lucene index and create embedded representation of it. Then you can search over that representation for "semantic" matches.

Last time I checked it about a year ago the embedded collection of documents was kept in the memory and the search was implemented by a linear scan. So I suspect it can be slow on very large collection of documents.

Re: LexVec, a word embedding model written in Go that outperforms word2vec

#45

Post by argonaut »

rspeer wrote: post #37
argonaut wrote: post #36

Earlier quoted context omitted.

People avoid ensembles, unless they're trying to squeeze out higher performance on a benchmark (or they're doing research on the technique of ensembling in of itself). I would have thought this would be obvious to you, since you're a researcher.

My company, Luminoso, uses Conceptnet Numberbatch as one component for interpreting the meaning of text. When the benchmarks went up, the understandability of its results did too. You get better search results, better topics, clearer visualizations. I'm not just trying to squeeze out extra performance, I'm trying to make computers understand text better. The benchmarks are the evidence that it's better. I do consider…

This is all good. It justifies a technical report, a blog post, a workshop paper, or publication in venues looking for this kind of owrk.

It doesn't necessarily justify publication in a venue looking for totally new ideas.

Post reply on HN