Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Semantic Grep – A Word2Vec-powered search tool

github.com

Post reply on HN
Print view View original

41–50 of 58 posts

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#41

Post by neonsunset »

microtonal wrote: post #31

Some small tips from superficially reading the code: https://github.com/arunsupe/semantic-grep/blob/b7dcc82a7cbab... You can read the vector all at once. See e.g.: https://github.com/danieldk/go2vec/blob/ee0e8720a8f518315f35... --- https://github.com/arunsupe/semantic-grep/blob/b7dcc82a7cbab... You can compute the similarity much faster by using BLAS. Good BLAS libraries have SIMD-optimized implementations. Or if you…

SIMD situation in Go is still rather abysmal, it’s likely easier to just FFI (though FFI is very slow so I guess you're stuck with ugly go asm if you are using short vectors). As usual, there's a particular high-level language that does it very well, and has standard vector similarity function nowadays...

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#42

Post by djoldman »

throw156754228 wrote: post #38

The model of a word to a vector breaks down really quickly one you introduce the context and complexity of human language. That's why we went to contextual embeddings, but even they have issues. Curious would it handle negation of trained keywords, e.g "not urgent"?

Definitely a limitation of word2vec as applied here.

Something like SBERT addresses this.

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#43

Post by ttpphd »

rldjbpin wrote: post #35

document search, an llm use-case almost all companies want for their white-collar workers, currently follows creation of a RAG. vector search is the first step of the process, and i argue that getting top n results and letting the user process them is probably the best of both worlds without involving "AI". this can be enhanced with multilingual support based on the encoder used. but building the index is still an ex…

You might be interested in Semantra: https://github.com/freedmand/semantra

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#44

Post by bitexploder »

neonsunset wrote: post #41
microtonal wrote: post #31

Some small tips from superficially reading the code: https://github.com/arunsupe/semantic-grep/blob/b7dcc82a7cbab... You can read the vector all at once. See e.g.: https://github.com/danieldk/go2vec/blob/ee0e8720a8f518315f35... --- https://github.com/arunsupe/semantic-grep/blob/b7dcc82a7cbab... You can compute the similarity much faster by using BLAS. Good BLAS libraries have SIMD-optimized implementations. Or if you…

SIMD situation in Go is still rather abysmal, it’s likely easier to just FFI (though FFI is very slow so I guess you're stuck with ugly go asm if you are using short vectors). As usual, there's a particular high-level language that does it very well, and has standard vector similarity function nowadays...

Go ahead and share the language, it is good etiquette for HN :)

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#45

Post by bitexploder »

throwaway290 wrote: post #34
tracerbulletx wrote: post #28

Earlier quoted context omitted.

God, the amount of enterprise speak in there is overwhelming.

Because of how it is marketed. But it is real and very much works.

And you can just go to its github to avoid the product side.

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#46

Post by neonsunset »

bitexploder wrote: post #44
neonsunset wrote: post #41

Earlier quoted context omitted.

SIMD situation in Go is still rather abysmal, it’s likely easier to just FFI (though FFI is very slow so I guess you're stuck with ugly go asm if you are using short vectors). As usual, there's a particular high-level language that does it very well, and has standard vector similarity function nowadays...

Go ahead and share the language, it is good etiquette for HN :)

Actually, there's three:

C#: https://github.com/dotnet/runtime/blob/main/docs/coding-guid... (and full family of other types: Vector2/3/4, Matrix3x2/4x4 and upcoming Tensor), vector similarity I was talking about is this: https://learn.microsoft.com/en-us/dotnet/api/system.numerics..., it uses a highly optimized SIMD kernel, for DotProduct just use an adjacent method

Swift: https://developer.apple.com/documentation/swift/simd-vector-... it is also a competent language at portable SIMD by virtue of using LLVM and offering almost the same operators-based API (e.g. masked = vec1 & ~vec2) like C#

Mojo: https://docs.modular.com/mojo/stdlib/builtin/simd which follows the above two, it too targets LLVM so expect good SIMD codegen as long the lowering strategy does it in an LLVM-friendly way, which I have not looked at yet.

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#47

Post by VHRanger »

throw156754228 wrote: post #38

The model of a word to a vector breaks down really quickly one you introduce the context and complexity of human language. That's why we went to contextual embeddings, but even they have issues. Curious would it handle negation of trained keywords, e.g "not urgent"?

For a lot of cases, word2vec/glove still work plenty well. It also runs much faster and lighter when doing development -- the FSE library [1] does 0.5M sentences/sec on CPU, whereas the fastest sentence_transformers [2] do something like 20k sentences/sec on a V100 (!!).

For the drawbacks:

Word embeddings are only good at similarity search style queries - stuff like paraphrasing.

Negation they'll necessarily struggle with. Since word embeddings are generally summed or averaged into a sentence embedding, a negation won't shift the sentence vector space around the way it would in a LM embedding.

Also things like homonyms are issues, but this is massively overblown as a reason to use LM embeddings (at least for latin/germanic languages).

Most people use LM embeddings because they've been told it's the best thing by other people rather than benchmarking accuracy and performance for their usecase.

1. https://github.com/oborchers/Fast_Sentence_Embeddings

2. https://www.sbert.net/docs/sentence_transformer/pretrained_m...

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#48

Post by ignoramous »

throwawaydummy wrote: post #21
ignoramous wrote: post #19

Earlier quoted context omitted.

Mistral has published large language models, not embedding models? sgrep uses Google's Word2Vec to generate embeddings of the corpus and perform similarity searches on it, given a user query.

No I got that I asked because wouldn’t embedding generated by fine tuned transformer based LLMs be more context aware? Idk much about the internals so apologies if this was a dumb thing to say

embeddings come in handy to augment LLMs [0], but as you suspect, some try LLMs themselves as an outright embedding model with varying degrees of success: https://www.reddit.com/r/LocalLLaMA/comments/12y3stx/embeddi... / https://huggingface.co/spaces/mteb/leaderboard

[0] https://simonwillison.net/2023/Oct/23/embeddings/

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#49

Post by e12e »

neonsunset wrote: post #46
bitexploder wrote: post #44

Earlier quoted context omitted.

Go ahead and share the language, it is good etiquette for HN :)

Actually, there's three: C#: https://github.com/dotnet/runtime/blob/main/docs/coding-guid... (and full family of other types: Vector2/3/4, Matrix3x2/4x4 and upcoming Tensor ), vector similarity I was talking about is this: https://learn.microsoft.com/en-us/dotnet/api/system.numerics... , it uses a highly optimized SIMD kernel, for DotProduct just use an adjacent method Swift: https://developer.apple.com/documentation…

Not Julia?

Re: Show HN: Semantic Grep – A Word2Vec-powered search tool

#50

Post by neonsunset »

e12e wrote: post #49
neonsunset wrote: post #46

Earlier quoted context omitted.

Actually, there's three: C#: https://github.com/dotnet/runtime/blob/main/docs/coding-guid... (and full family of other types: Vector2/3/4, Matrix3x2/4x4 and upcoming Tensor ), vector similarity I was talking about is this: https://learn.microsoft.com/en-us/dotnet/api/system.numerics... , it uses a highly optimized SIMD kernel, for DotProduct just use an adjacent method Swift: https://developer.apple.com/documentation…

Not Julia?

Julia is less "general-purpose" and I know little about the quality of its codegen (it does target LLVM but I haven't seen numbers that place it exactly next to C or C++ which is the case with C#).

Mojo team's blog posts do indicate they care about optimal compiler output, and it seems to have ambitions for a wider domain of application which is why it is mentioned.

Post reply on HN