Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Choosing vector database: a side-by-side comparison

benchmark.vectorview.ai

Post reply on HN
Print view View original

41–50 of 125 posts

Re: Choosing vector database: a side-by-side comparison

#41

Post by emilfroberg »

andre-z wrote: post #21
emilfroberg wrote: post #2

I made this table to compare vector databases in order to help me choose the best one for a new project. I spent quite a few hours on it, so I wanted to share it here too in hopes it might help others as well. My main criteria when choosing vector DB were the speed, scalability, dx, community and price. You'll find all of the comparison parameters in the article.

I'm curious where got the numbers on qps? They are pretty different from our experience. Reached out on LinkedIn. ;)

Happy to connect. The benchmark numbers are mostly from ANN Benchmarks. For my use case, the nytimes-256 dataset was most relevant so I used that for the QPS benchmark. I also took a look at the benchmarks you've made at https://qdrant.tech/benchmarks/ and there qdrant seems to be outperforming many others. If I've gotten something wrong here, I'm glad to update the article :)

Re: Choosing vector database: a side-by-side comparison

#42

Post by dmezzetti »

emilfroberg wrote: post #37
dmezzetti wrote: post #9

I'll add txtai to the list: https://github.com/neuml/txtai txtai is an all-in-one embeddings database for semantic search, LLM orchestration and language model workflows. Embeddings databases are a union of vector indexes (sparse and dense), graph networks and relational databases. This enables vector search with SQL, topic modeling and retrieval augmented generation. txtai adopts a local-first approach. A production…

Txtai looks interesting, maybe you could help me collect some of the comparision parameters for it?

Sure, I'd be happy to do so. Easiest way is probably using the public Slack channel that's accessible via the GitHub page.

Re: Choosing vector database: a side-by-side comparison

#43

Post by [deleted] »

softwaredoug wrote: post #10

Everyone I talk to who is building some vector db based thing sooner or later realizes they also care about the features of a full-text search engine. They care about filtering, they care to some degree about direct lexical matches, they care about paging, getting groups / facet counts, etc. Vectors, IMO, are just one feature that a regular search engine should have. IMO currently Vespa does the best job of this, tho…

[deleted]

Re: Choosing vector database: a side-by-side comparison

#44

Post by vosper »

softwaredoug wrote: post #10

Everyone I talk to who is building some vector db based thing sooner or later realizes they also care about the features of a full-text search engine. They care about filtering, they care to some degree about direct lexical matches, they care about paging, getting groups / facet counts, etc. Vectors, IMO, are just one feature that a regular search engine should have. IMO currently Vespa does the best job of this, tho…

My company is using vector search with Elasticsearch. It’s working well so far. IMO Elastic will eat most vector-first/only products because of its strength at full-text search, plus all the other stuff it does.

Re: Choosing vector database: a side-by-side comparison

#45

Post by deepsquirrelnet »

softwaredoug wrote: post #10

Everyone I talk to who is building some vector db based thing sooner or later realizes they also care about the features of a full-text search engine. They care about filtering, they care to some degree about direct lexical matches, they care about paging, getting groups / facet counts, etc. Vectors, IMO, are just one feature that a regular search engine should have. IMO currently Vespa does the best job of this, tho…

Until very recently, “dense retrieval” was not even as good as bm25, and still is not always better.

I think a lot of people use dense retrieval in applications where sparse retrieval is still adequate and much more flexible, because it has the hype behind it. Hybrid approaches also exist and can help balance the strengths and weaknesses of each.

Vectors can also work in other tasks, but largely people seem to be using them for retrieval only, rather than applying them to multiple tasks.

Re: Choosing vector database: a side-by-side comparison

#46

Post by citruscomputing »

Strongly disagree with PGVector's DX being worse than Chroma. Installing, configuring, and working with Chroma was infuriating -- it's alpha software and has the bugs and rough edges to prove it. The tools to support and interface with postgres are battle-tested and so much nicer by comparison; getting Chroma working took over a week, ripping it out and replacing with PGVector took a couple hours.

Also agree with this[0] article that vector search is only one type of search, and even for RAG isn't necessarily the one you want to start with.

[0]: https://colinharman.substack.com/p/beware-tunnel-vision-in-a...

Re: Choosing vector database: a side-by-side comparison

#47

Post by deepsquirrelnet »

What advantage are vector databases providing above using an index in conjunction with a mature database? I’m not sold on this as a separate technology.

Vector search is useful, but I don’t understand why I would go out of my way when I could implement FAISS or HNSWlib as an adjunct to postgres or a document store.

Re: Choosing vector database: a side-by-side comparison

#48

Post by spullara »

deepsquirrelnet wrote: post #47

What advantage are vector databases providing above using an index in conjunction with a mature database? I’m not sold on this as a separate technology. Vector search is useful, but I don’t understand why I would go out of my way when I could implement FAISS or HNSWlib as an adjunct to postgres or a document store.

Vector extensions to your current database or search engine makes far more sense than adding yet another dependency to manage and operate. The vector database folks will have to become a real database or full featured search engine to survive and compete with the incumbents that will all have good solutions for vector similarity search.

Re: Choosing vector database: a side-by-side comparison

#49

Post by fzliu »

citruscomputing wrote: post #46

Strongly disagree with PGVector's DX being worse than Chroma. Installing, configuring, and working with Chroma was infuriating -- it's alpha software and has the bugs and rough edges to prove it. The tools to support and interface with postgres are battle-tested and so much nicer by comparison; getting Chroma working took over a week, ripping it out and replacing with PGVector took a couple hours. Also agree with thi…

Shameless self-plug for milvus-lite:

   $ pip install milvus
   $ python
   >>> import milvus
   >>> milvus.start()

Re: Choosing vector database: a side-by-side comparison

#50

Post by dmezzetti »

deepsquirrelnet wrote: post #47

What advantage are vector databases providing above using an index in conjunction with a mature database? I’m not sold on this as a separate technology. Vector search is useful, but I don’t understand why I would go out of my way when I could implement FAISS or HNSWlib as an adjunct to postgres or a document store.

If you're interested in an approach like this, take a look at txtai.

1. https://neuml.github.io/txtai/embeddings/indexing/

2. https://neuml.hashnode.dev/external-database-integration

Post reply on HN