Why not use GPT 4 embeddings instead of word2vec ? Won't that be more effective
Show HN: Semantic Grep – A Word2Vec-powered search tool
51–58 of 58 posts
Re: Show HN: Semantic Grep – A Word2Vec-powered search tool
#52The model of a word to a vector breaks down really quickly one you introduce the context and complexity of human language. That's why we went to contextual embeddings, but even they have issues. Curious would it handle negation of trained keywords, e.g "not urgent"?
This implementation does not because the query has to be a word. One way to extend it to phrases is to average the vectors of each word in the phrase. Another way is to have a word2vec model that embeds phrases. (The large Google News model I think does have phrases, with spaces converted to underscore). But going from words to phrases opens up a whole new can of worms - how large a phrase to consider from the input…
Yep, it would almost be easier to:
- check to see if the search query contains more than one word
- spin up a more modestly sized LLM, such as mistral 7b
- ask the LLM to try to condense / find single word synonym for the user query
- send to sgrep
Re: Show HN: Semantic Grep – A Word2Vec-powered search tool
#53Why not use GPT 4 embeddings instead of word2vec ? Won't that be more effective
I would assume the argument could be made that "can be run locally", "w2v can be trained faster than other models" and "doesn't cost per-call to use" to be some arguments for why someone would try this.
Re: Show HN: Semantic Grep – A Word2Vec-powered search tool
#54Very nice ! How might one go about adapting this to other languages ? Does a version of the model downloaded exist somewhere ?
I would think translation to other languages would be trivial. The model is just a map of word to vector. Every word is converted to it's vector representation; then the query word is compared to the input words using cosine similarity.
I assumed you meant computer languages :-) If you mean human languages, yes Google publishes word2vec embeddings in many different human languages. Not sure though how easy it is to download.
Re: Show HN: Semantic Grep – A Word2Vec-powered search tool
#55just played around with it, not very smart per se, to get full power of semantic grep, LLM still might be needed, how to do it? is RAG the only way?
Re: Show HN: Semantic Grep – A Word2Vec-powered search tool
#56Hello Arun. Does it use cosine similarity for the matching?
Re: Show HN: Semantic Grep – A Word2Vec-powered search tool
#57Hello Arun. Does it use cosine similarity for the matching?
Correct.
Re: Show HN: Semantic Grep – A Word2Vec-powered search tool
#58Very nice ! How might one go about adapting this to other languages ? Does a version of the model downloaded exist somewhere ?
Added support for multiple languages, using fasttext's embeddings