Vectors are over, hashes are the future
algolia.com
Vectors are over, hashes are the future
1–10 of 43 posts
Re: Vectors are over, hashes are the future
#2Seems the author is proposing LSH instead of vectors for doing ANN?
There are benchmarks here, http://ann-benchmarks.com/ , but LSH underperforms the state of the art ANN algorithms like HNSW on recall/throughput.
LSH I believe was state of the art 10ish years ago, but has since been surpassed. Although the caching aspect is really nice.
Re: Vectors are over, hashes are the future
#3So my interpretation of the neural hash approach is largely that it is essentially trading a much larger number of very small “neurons” vs a smaller number of floats. Given that I’d be curious about what the total size difference is.
I could see the hash approach at a functional level resulting in different features essentially getting a different number of bit directly, which be approximately equivalent to having a NN with variable precision floats, all in a very hand wavy way.
Eg we could say a NN/NH needs N bits of information to work accurately, in which case you’re trading the format and operations on those Nbits
Re: Vectors are over, hashes are the future
#4This is a rehash (pardon me) of this post from 2021: https://www.search.io/blog/vectors-versus-hashes
The demand for vector embedding models (like those released by OpenAI, Cohere, HuggingFace, etc) and vector databases (like https://pinecone.io -- disclosure: I work there) has only grown since then. The market has decided that vectors are not, in fact, over.
Re: Vectors are over, hashes are the future
#5Hashes are just short, constrained membership vectors
Re: Vectors are over, hashes are the future
#6Seems the author is proposing LSH instead of vectors for doing ANN? There are benchmarks here, http://ann-benchmarks.com/ , but LSH underperforms the state of the art ANN algorithms like HNSW on recall/throughput. LSH I believe was state of the art 10ish years ago, but has since been surpassed. Although the caching aspect is really nice.
LSH is a _technique_, whose performance vastly/mostly depends on the hashing function and on how this function enables neighborhood exploration.
It might not be trendy, but it doesn't mean it can't work as good or better than HNSW. It all depends on the hashing function you come up with.
Re: Vectors are over, hashes are the future
#7Seems the author is proposing LSH instead of vectors for doing ANN? There are benchmarks here, http://ann-benchmarks.com/ , but LSH underperforms the state of the art ANN algorithms like HNSW on recall/throughput. LSH I believe was state of the art 10ish years ago, but has since been surpassed. Although the caching aspect is really nice.
10? no, it's more like 20+. lsh was a core piece of the google crawler. it was used for high performance fuzzy deduplication.
see ullman's text: mining massive datasets. it's free on the web.
Re: Vectors are over, hashes are the future
#8Seems the author is proposing LSH instead of vectors for doing ANN? There are benchmarks here, http://ann-benchmarks.com/ , but LSH underperforms the state of the art ANN algorithms like HNSW on recall/throughput. LSH I believe was state of the art 10ish years ago, but has since been surpassed. Although the caching aspect is really nice.
LSH is a _technique_, whose performance vastly/mostly depends on the hashing function and on how this function enables neighborhood exploration. It might not be trendy, but it doesn't mean it can't work as good or better than HNSW. It all depends on the hashing function you come up with.
when combined with minhashing it approximates jaccard similarity, so it seems it would be bounded by that.
Re: Vectors are over, hashes are the future
#9Hashes are fine, but to say that "vectors are over" is just plain nonsense. We continue to see vectors as a core part of production systems for entity representation and recommendation (example: https://slack.engineering/recommend-api) and within models themselves (example: multimodal and diffusion models). For folks into metrics, we're building a vector database specifically for storing, indexing, and searching across massive quantities of vectors (https://github.com/milvus-io/milvus), and we've seen close to exponential growth in terms of total downloads.
Vectors are just getting started.
Re: Vectors are over, hashes are the future
#10Seems the author is proposing LSH instead of vectors for doing ANN? There are benchmarks here, http://ann-benchmarks.com/ , but LSH underperforms the state of the art ANN algorithms like HNSW on recall/throughput. LSH I believe was state of the art 10ish years ago, but has since been surpassed. Although the caching aspect is really nice.
10? no, it's more like 20+. lsh was a core piece of the google crawler. it was used for high performance fuzzy deduplication. see ullman's text: mining massive datasets. it's free on the web.
I think LSH was only introduced in 99 by Indyk et. al. I would say it was a pretty active research area 10 years ago.