Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Vectors are over, hashes are the future

algolia.com

Post reply on HN
Print view View original

21–30 of 43 posts

Re: Vectors are over, hashes are the future

#21

Post by kvathupo »

nelsondev wrote: post #2

Seems the author is proposing LSH instead of vectors for doing ANN? There are benchmarks here, http://ann-benchmarks.com/ , but LSH underperforms the state of the art ANN algorithms like HNSW on recall/throughput. LSH I believe was state of the art 10ish years ago, but has since been surpassed. Although the caching aspect is really nice.

To elaborate on Noe's comment, the article is suggesting the use of LSH where the hashing function is learned by a neural network such that similar vectors correspond to similar hashes via Hamming weight (whilst enforcing some load factor). In effect, a good hash is generated by a neural network. It appears Elastiknn a prioi chooses the hash function? Not sure, not my area of knowledge.

This approach seems feasible tbh. For example, a stock's historical bids/asks probably don't deviate greatly from month to month. That said, the generation of a good hash is dependent on the stock ticker, and a human doesn't have the time to find a good one for every stock at scale.

Re: Vectors are over, hashes are the future

#22

Post by packetlost »

gk1 wrote: post #4

This is a rehash (pardon me) of this post from 2021: https://www.search.io/blog/vectors-versus-hashes The demand for vector embedding models (like those released by OpenAI, Cohere, HuggingFace, etc) and vector databases (like https://pinecone.io -- disclosure: I work there) has only grown since then. The market has decided that vectors are not, in fact, over.

PineCone seems interesting. Is the storage backend open source? I've been working on a persistent hashmap database that's somewhat similar (albeit not done) that should have less RAM requirements than bitcask (ie. larger than RAM keysets)

Re: Vectors are over, hashes are the future

#23

Post by kvathupo »

fzliu wrote: post #9

Hashes are fine, but to say that "vectors are over" is just plain nonsense. We continue to see vectors as a core part of production systems for entity representation and recommendation (example: https://slack.engineering/recommend-api ) and within models themselves (example: multimodal and diffusion models). For folks into metrics, we're building a vector database specifically for storing, indexing, and searching acr…

Click-bait title aside : ^ ), I'd agree. Neural hashes seem to be a promising advancement imo, but I question its impact on the convergence time of AI models. In the pecking order of neural network bottlenecks, I'd imagine it's not terribly expensive to access training data from some database. Rather, hardware considerations for improving parallelism seem to be the biggest hurdle [1].

[1] - https://www.nvidia.com/en-us/data-center/nvlink/

Re: Vectors are over, hashes are the future

#24

Post by mrkeen »

> The analogy here would be the choice between a 1 second flight to somewhere random in the suburb of your choosing in any city in the world versus a 10 hour trip putting you at the exact house you wanted in the city of your choice.

Wouldn't the first part of the analogy actually be:

A 1 second flight that will probably land at your exact destination, but could potentially land you anywhere on earth?

Re: Vectors are over, hashes are the future

#27

Post by gauddasa »

fzliu wrote: post #9

Hashes are fine, but to say that "vectors are over" is just plain nonsense. We continue to see vectors as a core part of production systems for entity representation and recommendation (example: https://slack.engineering/recommend-api ) and within models themselves (example: multimodal and diffusion models). For folks into metrics, we're building a vector database specifically for storing, indexing, and searching acr…

True. The title is just clickbait and what we find inside is suggestions for dimensionality reduction by a person who appears to be on the verge of reinventing autoencoders disguised as neural hashes. Is it a mere coincidence that the article fails to mention autoencoders?

Re: Vectors are over, hashes are the future

#28

Post by auraham »

whatever1 wrote: post #25

Omg NN “research” is just heuristics on top of heuristics on top of mambo jumbo. Hopefully someone who knows math will enter the field one day and build the theoretical basis for all this mess and allow us to make real progress.

Old post of Yann LeCun [1]:

> But another important goal is inventing new methods, new techniques, and yes, new tricks. In the history of science and technology, the engineering artifacts have almost always preceded the theoretical understanding: the lens and the telescope preceded optics theory, the steam engine preceded thermodynamics, the airplane preceded flight aerodynamics, radio and data communication preceded information theory, the computer preceded computer science.

[1] https://www.reddit.com/r/MachineLearning/comments/7i1uer/n_y...

Post reply on HN