Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Choosing vector database: a side-by-side comparison

benchmark.vectorview.ai

Post reply on HN
Print view View original

111–120 of 125 posts

Re: Choosing vector database: a side-by-side comparison

#111

Post by m00x »

charcircuit wrote: post #86
m00x wrote: post #66

Earlier quoted context omitted.

They have beef with ES since they took the software, made a bunch of cash on it, then never contributed back. ES called them out and it started a feud. I'd go on ES over Amazon-built software any day. I worked on RDS and I've used RDS at several companies, it's a mess. Longer story: One day one of our table went missing on Aurora, we couldn't figure out why, it was in the schema, etc. Devops panicked and restarted th…

>then never contributed back Amazon did contribute back.

I haven't kept up since the drama, it's possible they did after.

Re: Choosing vector database: a side-by-side comparison

#112

Post by bayesian_limit »

Havoc wrote: post #72

16x difference between pg and milvus? I thought for most use cases this would be quite performance sensitive

We conducted benchmark tests on Elastic's queries per second (QPS) performance using datasets of 500,000 and 1 million vectors. Result was Zilliz is 13x and 22x faster, per number of vectors respectively. https://zilliz.com/blog/elasticsearch-cloud-vs-zilliz

We also conducted a benchmark comparing Pgvector to both Milvus (open source) and Zilliz (managed, with a free tier option). When running the OSS Milvus on 2 CPUs and 8 GiB memory, Pgvector was found to be 5 times slower. You can check out the detailed performance charts at the bottom of this blog post: https://zilliz.com/blog/getting-started-pgvector-guide-devel...

Feel free to explore our open-source benchmarking tool, which allows you to examine our methodology and even compare it with your vector database. https://github.com/zilliztech/VectorDBBench

Re: Choosing vector database: a side-by-side comparison

#113

Post by Havoc »

bayesian_limit wrote: post #112
Havoc wrote: post #72

16x difference between pg and milvus? I thought for most use cases this would be quite performance sensitive

We conducted benchmark tests on Elastic's queries per second (QPS) performance using datasets of 500,000 and 1 million vectors. Result was Zilliz is 13x and 22x faster, per number of vectors respectively. https://zilliz.com/blog/elasticsearch-cloud-vs-zilliz We also conducted a benchmark comparing Pgvector to both Milvus (open source) and Zilliz (managed, with a free tier option). When running the OSS Milvus on 2 CPU…

Thanks. Pity I would have picked pg just because it would be less to learn

Re: Choosing vector database: a side-by-side comparison

#114

Post by freilanzer »

dathinab wrote: post #65
esafak wrote: post #52

Earlier quoted context omitted.

What do you like about it relative to alternatives? How fast is it?

much more mature and feature rich then many of the competition listed in the article to some degree it's more a platform you can use to efficiently and flexible build your own more complicated search system, which is both a benefit and drawback some good parts: - very flexible text search (bm25), more so then elastic search (or at least easier to user/better documented when it comes to advanced features) - fast flexi…

> - multiple vectors per document

Can I have (multiple) vectors for a single field? That would be quite helpful.

Re: Choosing vector database: a side-by-side comparison

#115

Post by freilanzer »

totalhack wrote: post #93

I'm interested to try some of these others next time around, but I've used qdrant self-hosted in two projects and been pleased. Milvus was recommended so I gave that a try but found it over complicated. Pgvector seems like an obvious choice if you are already using postgres and if that performance is ok.

Overly complicated in what way exactly?

Re: Choosing vector database: a side-by-side comparison

#116

Post by jkb79 »

freilanzer wrote: post #114
dathinab wrote: post #65

Earlier quoted context omitted.

much more mature and feature rich then many of the competition listed in the article to some degree it's more a platform you can use to efficiently and flexible build your own more complicated search system, which is both a benefit and drawback some good parts: - very flexible text search (bm25), more so then elastic search (or at least easier to user/better documented when it comes to advanced features) - fast flexi…

> - multiple vectors per document Can I have (multiple) vectors for a single field? That would be quite helpful.

Yes, Vespa has a generic Tensor framework that allows you to index multiple vectors for a single field, see https://blog.vespa.ai/semantic-search-with-multi-vector-inde... for details.

field embeddings type tensor(p{}, x[384]) to represent a multi-vector field { "0": [0.1....], "1": [0.2,..] }

Re: Choosing vector database: a side-by-side comparison

#117

Post by redwood »

noonething wrote: post #99
softwaredoug wrote: post #10

Everyone I talk to who is building some vector db based thing sooner or later realizes they also care about the features of a full-text search engine. They care about filtering, they care to some degree about direct lexical matches, they care about paging, getting groups / facet counts, etc. Vectors, IMO, are just one feature that a regular search engine should have. IMO currently Vespa does the best job of this, tho…

How does Vespa even make money? Just cloud stuff? That's nuts if search is already built in.

They have traditionally been part of Yahoo and just spun out

Re: Choosing vector database: a side-by-side comparison

#118

Post by charliejuggler »

You might like the 'Which Search Engine?' panel I ran at Buzzwords earlier this year with some of the leading contenders (Vespa, Qdrant, Elastic, Solr, Weaviate) https://www.youtube.com/watch?v=iI40L4wMtyI - vector search was obviously part of the discussion

Re: Choosing vector database: a side-by-side comparison

#119

Post by dathinab »

freilanzer wrote: post #114
dathinab wrote: post #65

Earlier quoted context omitted.

much more mature and feature rich then many of the competition listed in the article to some degree it's more a platform you can use to efficiently and flexible build your own more complicated search system, which is both a benefit and drawback some good parts: - very flexible text search (bm25), more so then elastic search (or at least easier to user/better documented when it comes to advanced features) - fast flexi…

> - multiple vectors per document Can I have (multiple) vectors for a single field? That would be quite helpful.

yes that is what I meant

generally if you have multiple embeddings for the same document you have two choices:

- create one document for each embedding and make sure non membedding specific attributes are the same across all of this document clones -- vespa makes this more convenient by having child documents

- have a field with multiple documents, i.e. there are multipel vectors in the HNSW-index which point to the same document -- vespa support this, too. It's what I meant.

vespa is currently the only vector search enabled search system which supports both in a convenient way, but then there are so many "vector databases" poping up every month that I might have missed some

Re: Choosing vector database: a side-by-side comparison

#120

Post by totalhack »

freilanzer wrote: post #115
totalhack wrote: post #93

I'm interested to try some of these others next time around, but I've used qdrant self-hosted in two projects and been pleased. Milvus was recommended so I gave that a try but found it over complicated. Pgvector seems like an obvious choice if you are already using postgres and if that performance is ok.

Overly complicated in what way exactly?

It was a while ago now so the details have faded, but for one all of the docker services it had to spin up vs the single container that qdrant runs. I'm sure there is a reason for this, but I haven't needed it.
Post reply on HN