K-Nearest Neighbors • Hacker News Board

Re: K-Nearest Neighbors

#31

Post by generall » Wed, Apr 13, 2022, 11:59 PM UTC

turnersr wrote: post #24

gk1 wrote: post #23

Earlier quoted context omitted.

Pinecone stores and searches through dense vector embeddings using a proprietary ANN index. It also has live index updates and metadata filtering, which you’d expect from any database but is surprisingly hard to find or do with vector indexes. As you said, common use cases include deduplication and image search, and especially semantic search (text).

Do you happen to know other implementations that allow for live updates and metadata filtering like Pinecone?

Check out https://github.com/qdrant/qdrant

Re: K-Nearest Neighbors

#32

Post by redskyluan » Thu, Apr 14, 2022, 3:17 AM UTC

oofbey wrote: post #28

Why would somebody use pinecone instead of a high quality open source library like FAISS？

FAISS is simply a library. You will need a system Like Milvus for huge data amount

Re: K-Nearest Neighbors

#33

Post by redskyluan » Thu, Apr 14, 2022, 3:19 AM UTC

turnersr wrote: post #24

gk1 wrote: post #23

Earlier quoted context omitted.

Pinecone stores and searches through dense vector embeddings using a proprietary ANN index. It also has live index updates and metadata filtering, which you’d expect from any database but is surprisingly hard to find or do with vector indexes. As you said, common use cases include deduplication and image search, and especially semantic search (text).

Do you happen to know other implementations that allow for live updates and metadata filtering like Pinecone?

See https://milvus.io/

Re: K-Nearest Neighbors

#34

Post by rgavuliak » Thu, Apr 14, 2022, 6:21 AM UTC

kNN is very well understood and well documented. It seems though someone makes a blog post about it every week that gets upvoted on HN. This only further solidifies the notion that most blogs relating to Data Science rarely cover advanced topics.

Re: K-Nearest Neighbors

#35

Post by seanhunter » Thu, Apr 14, 2022, 7:44 AM UTC

mrfusion wrote: post #4

antognini wrote: post #3

k-NN is kind of an interesting machine learning algorithm because it is extremely simple (just take the average of the closest k data points in your dataset), but is, in a certain sense, the "ideal" algorithm. In the limit that the size of your data goes to infinity, k-NN performs perfectly. With the appropriate choice of k it is impossible for any algorithm, no matter how clever, to outperform it. Other ML algorithm…

I believe it also makes overtraining impossible which is pretty strange for a learning algorithm.

I don't think that's the case. Depending on your dataset and objective function, for some (too small) values of k, kNN will settle into local minima and not locate the globally optimal solution (I believe).

Re: K-Nearest Neighbors

#36

Post by Mandelmus » Thu, Apr 14, 2022, 7:52 AM UTC

rgavuliak wrote: post #34

kNN is very well understood and well documented. It seems though someone makes a blog post about it every week that gets upvoted on HN. This only further solidifies the notion that most blogs relating to Data Science rarely cover advanced topics.

I feel like the percentage of "Let me make a blog post or YouTube tutorial about this thing I learned about yesterday" content on technical topics is trending towards 100% as time goes on.

Re: K-Nearest Neighbors

#37

Post by jamesbriggs » Thu, Apr 14, 2022, 8:58 AM UTC

oofbey wrote: post #28

Why would somebody use pinecone instead of a high quality open source library like FAISS？

Faiss is great, but I see there being several strong reasons for using Pinecone:

1. The people who built the Pinecone vector index include some of the top experts in information retrieval/vector search in the world, the chances of someone like myself building something more efficient in Faiss is pretty slim, even after spending months learning the library and as much as I can about vector search

2. Faiss is only a vector index, it doesn't manage your data, so you need to personally build a framework around your Faiss index to handle that, speaking from experience this can be frustrating

3. Pinecone has one of (if not the) most advanced implementation of filtering available, they use 'single stage' filtering, which provides pre-filtering speeds with post-filtering accuracy

4. Scalability, want to search through an index of 2B vectors? Not an issue, getting that working with Faiss is not fun

5. Ease of use, all of the above takes an API key and ~5 lines of code to setup

Re: K-Nearest Neighbors

#38

Post by rgavuliak » Thu, Apr 14, 2022, 2:15 PM UTC

Mandelmus wrote: post #36

rgavuliak wrote: post #34

kNN is very well understood and well documented. It seems though someone makes a blog post about it every week that gets upvoted on HN. This only further solidifies the notion that most blogs relating to Data Science rarely cover advanced topics.

I feel like the percentage of "Let me make a blog post or YouTube tutorial about this thing I learned about yesterday" content on technical topics is trending towards 100% as time goes on.

Exactly, I know the advice to boost your resume often tells you to write a blogpost or do a talk, but I would also say that you should do this if you have something interesting to say. Until that happens you're probably better off improving your technical skills.