21–30 of 95 posts
I work at Zilliz ( https://zilliz.com ) and am a part of the Milvus community. Here are some other resources in case anybody's interested in learning more about embeddings, vector search, and vector databases: 1) Embedding crash course https://developers.google.com/machine-learning/crash-course/... 2) What is a vector database? https://zilliz.com/learn/what-is-vector-database 3) Introduction to vector similarity sear…
Does Zilliz do BYOC?
Are they guaranteed to return the most optimal similar neighbors or will they be sloppy in return for efficiency?
I only heard about vector databases along with the recent advents of AI. Assuming they've been around for a while, what were the benefits of using them over "normal" search engines (e.g. ElasticSearch)?
Here's a highly simplified example: if my query was "fields related to computer science", a semantic engine could return "statistics" and "electrical engineering" while avoiding things such as "social science" and "political science".
What algorithms are these databases using for fast similarity search? Are they guaranteed to return the most optimal similar neighbors or will they be sloppy in return for efficiency?
The most commonly used one I've seen is HNSW, which greatly speeds up search at the expensive of some extra memory consumption. Recall is also strong - for many large-scale use cases you can get 95%+ with properly tuned parameters.
If anyone from AWS/Google/Azure is listening, please add pgvector [1] into your managed Postgres offerings!
I only heard about vector databases along with the recent advents of AI. Assuming they've been around for a while, what were the benefits of using them over "normal" search engines (e.g. ElasticSearch)?
It feels like there are an influx of "vector databases" right now, I haven't had a strong answer out of anyone on why you'd be better off using these over Redis which offers vector storage with similarity search in a battle-tested OSS solution.
I only heard about vector databases along with the recent advents of AI. Assuming they've been around for a while, what were the benefits of using them over "normal" search engines (e.g. ElasticSearch)?
ES has support for vector search now too. Really you want both in use cases where the user expects the the top results to contain the search keywords, but also wants results that are synonyms or conceptually similar. TF/IDF and BM25 help with first part and vectors help with the second. Theoretically only vectors should be needed, but that isn't my experience in practice.