Vectors are over, hashes are the future of AI
1–10 of 27 posts
Re: Vectors are over, hashes are the future of AI
#2Whoever wrote the article must have done a cursory search at best, I'm surprised they didn't mention semantic hashing by Salakhutdinov & Hinton (2007) https://www.cs.utoronto.ca/~rsalakhu/papers/semantic_final.p...
Edit: also, talking about LSH, must check out FAISS library https://github.com/facebookresearch/faiss and the current SOTA http://ann-benchmarks.com/
Re: Vectors are over, hashes are the future of AI
#3I feel like I have been talking about LSH for years
Re: Vectors are over, hashes are the future of AI
#4> "_If this peaked your interest_"
It didn't.[1]
[1]: https://www.merriam-webster.com/words-at-play/pique-vs-peak-...
Re: Vectors are over, hashes are the future of AI
#5Contrastive and triplet loss is pretty cool for generating hashes. I'd imagine the trick they are alluding to is a rewrite the loss function to be more aware of locality instead of trying to minimize/maximize distance.
Or they are just shingling different ML hash functions, which is kinda lazy.
Re: Vectors are over, hashes are the future of AI
#6hm. I'd like to believe, but the arguments here seem a bit obtuse.
No one measures vector distance using the hamming distance on binary representations. That's silly. We use L1 or L2, usually, and the binary encoding of the numbers is irrelevant.
It sounds like the LSH is maaaaybe equivalent to vector quantization. In which case this would be a form of regularization, which sometimes works well, and sometimes meh.
Re: Vectors are over, hashes are the future of AI
#7let's just take it to its logical extension and make every model just one big look up table (with hashes as keys). /s
Re: Vectors are over, hashes are the future of AI
#8I feel like I have been talking about LSH for years
I recall Uber using it in a white paper writeup.
Is the speedup that remarkable? I'd be curious at to the increase in speed versus loss of precision.
Re: Vectors are over, hashes are the future of AI
#9using fancy neural nets for learning hash functions from data is indeed pretty cool, but hash functions fit to data isn't new. see "perfect hash functions."
lsh is most famously used for approximating jaccard distances, which even if you're not doing stuff like looking at lengths or distances in l1 or l2, is still a vector operation.
lsh is best described in jeff ullman's mining massive datasets textbook (available free online), which describes how it was used for webpage deduplication in the early days at google.
Re: Vectors are over, hashes are the future of AI
#10hm. I'd like to believe, but the arguments here seem a bit obtuse. No one measures vector distance using the hamming distance on binary representations. That's silly. We use L1 or L2, usually, and the binary encoding of the numbers is irrelevant. It sounds like the LSH is maaaaybe equivalent to vector quantization. In which case this would be a form of regularization, which sometimes works well, and sometimes meh.
How would these ml predictions be explainable?