Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Semantic Grep – A Word2Vec-powered search tool

github.com

Post reply on HN
Print view View original

31–40 of 58 posts

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#31

Post by microtonal »

Some small tips from superficially reading the code:

https://github.com/arunsupe/semantic-grep/blob/b7dcc82a7cbab...

You can read the vector all at once. See e.g.:

https://github.com/danieldk/go2vec/blob/ee0e8720a8f518315f35...

---

https://github.com/arunsupe/semantic-grep/blob/b7dcc82a7cbab...

You can compute the similarity much faster by using BLAS. Good BLAS libraries have SIMD-optimized implementations. Or if you do multiple tokens as once, you can do a matrix-vector multiplication (sgemv), which will be even faster in many implementations. Alternatively, there is probably also a SIMD implementation in Go using assembly (it has been 7 years since I looked at anything in the Go ecosystem).

You could also normalize the vectors while loading. Then during runtime the cosine similarity is just the dot product of the vectors (whether it pays off depends on the size of your embedding matrix and the size of the haystack that you are going to search).

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#32

Post by arunsupe »

_flux wrote: post #30

I wonder if it would be possible to easily add support for multiple CPUs? It seems to be taking at most 150% CPU, so on my workstation it could be (assuming high parallellism) 10 times as fast. Alas the word2vec repository has reached its quota: fetch: Fetching reference refs/heads/master batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to r…

The model in Google drive is the official model from Google and will work.

Haven't tried the huggingface model, but, looks very different. Unlikely to work.

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#33

Post by 9dev »

michaelmior wrote: post #15
9dev wrote: post #7

Earlier quoted context omitted.

As someone working extensively with word2vec: I would recommend to set up Elasticsearch. It has support for vector embeddings, so you can process your PDF documents once, write the word2vec embeddings and PDF metadata into an index, and search that in milliseconds later on. Doing live vectorisation is neat for exploring data, but using Elasticsearch will be much more convenient in actual products!

I would personally vote for Postgres and one of the many vector indexing extensions over Elasticsearch. I think Elasticsearch can be more challenging to maintain. Certainly a matter of opinion though. Elasticsearch is a very reasonable choice.

Elasticsearch is a pain to maintain, the docs are all over the place, and the API is what you end up with if developers run free and implement everything that jumps to their mind.

But there just isn’t anything comparable when it comes to building your own search engine. Postgres with a vector extension is good if all you want to do is a vector search and some SQL (not dismissing it here, I love PG); but if you want more complex search cases, Elasticsearch is the way to go.

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#34

Post by throwaway290 »

tracerbulletx wrote: post #28
fzeindl wrote: post #25

Fyi, there is already a widely used tool along with a company called semgrep, which stems from semantic grep: https://semgrep.dev/ .

God, the amount of enterprise speak in there is overwhelming.

Because of how it is marketed. But it is real and very much works.

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#35

Post by rldjbpin »

document search, an llm use-case almost all companies want for their white-collar workers, currently follows creation of a RAG.

vector search is the first step of the process, and i argue that getting top n results and letting the user process them is probably the best of both worlds without involving "AI".

this can be enhanced with multilingual support based on the encoder used. but building the index is still an expensive process and i wonder how that could be done fast for a local user.

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#37

Post by arunsupe »

infruset wrote: post #36

Very nice ! How might one go about adapting this to other languages ? Does a version of the model downloaded exist somewhere ?

I would think translation to other languages would be trivial. The model is just a map of word to vector. Every word is converted to it's vector representation; then the query word is compared to the input words using cosine similarity.

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#38

Post by throw156754228 »

The model of a word to a vector breaks down really quickly one you introduce the context and complexity of human language. That's why we went to contextual embeddings, but even they have issues.

Curious would it handle negation of trained keywords, e.g "not urgent"?

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#40

Post by arunsupe »

throw156754228 wrote: post #38

The model of a word to a vector breaks down really quickly one you introduce the context and complexity of human language. That's why we went to contextual embeddings, but even they have issues. Curious would it handle negation of trained keywords, e.g "not urgent"?

This implementation does not because the query has to be a word. One way to extend it to phrases is to average the vectors of each word in the phrase. Another way is to have a word2vec model that embeds phrases. (The large Google News model I think does have phrases, with spaces converted to underscore). But going from words to phrases opens up a whole new can of worms - how large a phrase to consider from the input stream etc. Plus, I don't think averaging words in the phrase is the same as learning the embedding for the phrase. Sentence embedding models are necessary for that, but they are far too slow for this use case as pointed out by others.

To summarize, this is a simple implementation that works for the simplest use case - semantic matching of words.

Post reply on HN