Live data from Hacker News

Choosing vector database: a side-by-side comparison

benchmark.vectorview.ai

61–70 of 125 posts

Re: Choosing vector database: a side-by-side comparison

#61

Postgres vector store has been the most simple, and will be if you are at a lower scale. You can just use it directly with something like spring boot.

Agreed on pgvector being simple and a great choice for POCs and low scale, especially if you're familiar with Postgres. Our team released something new last week built for folks looking to use PostgreSQL at scale as a vector store [0], featuring a DiskANN index type.

[0]: https://www.timescale.com/blog/how-we-made-postgresql-the-be...

Re: Choosing vector database: a side-by-side comparison

#62
Their definition about Hybrid Search is I think wrong.

Through this terms tend to not be consistently defined at all, so "wrong" is maybe the wrong word.

Their definition seem to be about filtering results during (approximate) KNN vector search.

But that is filtering, not hybrid search. Through it might sometimes be implemented as a form of hybrid search, but that's an internal implementation detail and you probably should hope it's not implemented that way.

Hybrid search is when you do both a vector search and a more classical text based search (e.g. bm25) and combine both results in a reasonable way.

Re: Choosing vector database: a side-by-side comparison

#63
post #44

Everyone I talk to who is building some vector db based thing sooner or later realizes they also care about the features of a full-text search engine. They care about filtering, they care to some degree about direct lexical matches, they care about paging, getting groups / facet counts, etc. Vectors, IMO, are just one feature that a regular search engine should have. IMO currently Vespa does the best job of this, tho…

My company is using vector search with Elasticsearch. It’s working well so far. IMO Elastic will eat most vector-first/only products because of its strength at full-text search, plus all the other stuff it does.

it's also has tones of subtle issues and we are constantly looking for potential replacements

Re: Choosing vector database: a side-by-side comparison

#64

Everyone I talk to who is building some vector db based thing sooner or later realizes they also care about the features of a full-text search engine. They care about filtering, they care to some degree about direct lexical matches, they care about paging, getting groups / facet counts, etc. Vectors, IMO, are just one feature that a regular search engine should have. IMO currently Vespa does the best job of this, tho…

Until very recently, “dense retrieval” was not even as good as bm25, and still is not always better. I think a lot of people use dense retrieval in applications where sparse retrieval is still adequate and much more flexible, because it has the hype behind it. Hybrid approaches also exist and can help balance the strengths and weaknesses of each. Vectors can also work in other tasks, but largely people seem to be usi…

more commonly you use approximate KNN vector search with LLM based embeddings, which can find many fitting documents bm25 and similar would never manage to

the tricky part if to properly combine the results

Re: Choosing vector database: a side-by-side comparison

#65
post #52

Earlier quoted context omitted.

Vespa.ai does a great job. Absolutely stunning thing!

What do you like about it relative to alternatives? How fast is it?

much more mature and feature rich then many of the competition listed in the article

to some degree it's more a platform you can use to efficiently and flexible build your own more complicated search system, which is both a benefit and drawback

some good parts:

- very flexible text search (bm25), more so then elastic search (or at least easier to user/better documented when it comes to advanced features)

- fast flexible enough vector search, with good filtering capabilities

- build in support for defining more complicated search piplines, including multi phase search (also known as rerankin)

- quite nice approach for more fine controlling about what kind of indices are build for which fields

- when doing schema changes has safety checks to make sure you don't accidentally brake anything, which you can override if you are sure you want that

- ton of control in a cluster about where which search system resources get allocated (e.g. which schemas get stored on which storage clusters, which cluster nodes should act as storage nodes, which should e.g. only do preprocessing or post processing steps in a search piplines and which e.g. should be used for calculating embeddings using some LLM or similar) Not something you for demos but definitly something you need once you customers have enough data.

- child documents, and document references

- multiple vectors per document

- quite a interesting set of data types for fields and related ways you can use them in a search pipline

- an flexible reasonable easy to use system for plugins/extensions (through Java only)

- support building search piplines which have sub-searches in extern potentially non vespa systems

- really well documented

Through the main benefit *and drawback* is that it's not just a vector database, but a full fledged search system platform.

Re: Choosing vector database: a side-by-side comparison

#66
post #44

Earlier quoted context omitted.

My company is using vector search with Elasticsearch. It’s working well so far. IMO Elastic will eat most vector-first/only products because of its strength at full-text search, plus all the other stuff it does.

Amazon was already working on getting rid of ElasticSearch with their Kendra NLP search. Are you sure ElasticSearch has rosy future?

They have beef with ES since they took the software, made a bunch of cash on it, then never contributed back. ES called them out and it started a feud.

I'd go on ES over Amazon-built software any day. I worked on RDS and I've used RDS at several companies, it's a mess.

Longer story: One day one of our table went missing on Aurora, we couldn't figure out why, it was in the schema, etc. Devops panicked and restarted the instance, and then another table was missing. We ended up creating 10 empty tables and restarted it until it hit one of those.

We contacted RDS support after that, and the conclusion of their 3 month investigation is: "Yeah, it's not supposed to do that."

There's some really smart people working at Amazon, unfortunately the incentives is to push new stuff out and get promoted ASAP. If you can do that better than others and before your house of cards falls, you're safe. If the house of card crumbles after you're gone, it's their problem.

Re: Choosing vector database: a side-by-side comparison

#67

Everyone I talk to who is building some vector db based thing sooner or later realizes they also care about the features of a full-text search engine. They care about filtering, they care to some degree about direct lexical matches, they care about paging, getting groups / facet counts, etc. Vectors, IMO, are just one feature that a regular search engine should have. IMO currently Vespa does the best job of this, tho…

Agreed, vector search is great but it's only one of many tools you can use to create a great search solution.

We recently did a bunch of evaluation work to quantify the differences between keyword search, vector search, hybrid, reranking, etc. across a few datasets. We shared the results here: https://techcommunity.microsoft.com/t5/azure-ai-services-blo...

Disclosure - I work in the Azure Search team.

Re: Choosing vector database: a side-by-side comparison

#68
post #12

I don't think we need specialized databases for vectors. Relational databases can easily be expanded by vector data types and operations. They will eventually catch up by supporting what was once a unique feature of the new system: https://medium.com/@magda7817/two-things-to-keep-in-mind-bef...

Yeah, this is my sense too. They will be slower to add these new requirements but they should be able to add these vector capabilities within a year or so. It's then a question of ability of smaller vector db companies to mature and add regular db capabilities, while innovating.

Re: Choosing vector database: a side-by-side comparison

#69
post #49

Strongly disagree with PGVector's DX being worse than Chroma. Installing, configuring, and working with Chroma was infuriating -- it's alpha software and has the bugs and rough edges to prove it. The tools to support and interface with postgres are battle-tested and so much nicer by comparison; getting Chroma working took over a week, ripping it out and replacing with PGVector took a couple hours. Also agree with thi…

Shameless self-plug for milvus-lite: $ pip install milvus $ python >>> import milvus >>> milvus.start()

Gonna add some information here since this isn't very descriptive.

milvus-lite is a bit like sqlite where it runs in-process. Here are some scenarios you'd want to use it in:

- You want to use Milvus directly without having it installed using Milvus - Operator, Helm, or Docker Compose etc. - You do not want to launch any virtual machines or containers while you are using Milvus. - You want to embed Milvus features in your Python applications.

Re: Choosing vector database: a side-by-side comparison

#70
post #66

Earlier quoted context omitted.

Amazon was already working on getting rid of ElasticSearch with their Kendra NLP search. Are you sure ElasticSearch has rosy future?

They have beef with ES since they took the software, made a bunch of cash on it, then never contributed back. ES called them out and it started a feud. I'd go on ES over Amazon-built software any day. I worked on RDS and I've used RDS at several companies, it's a mess. Longer story: One day one of our table went missing on Aurora, we couldn't figure out why, it was in the schema, etc. Devops panicked and restarted th…

>Longer story: One day one of our table went missing on Aurora, we couldn't figure out why, it was in the schema, etc. Devops panicked and restarted the instance, and then another table was missing. We ended up creating 10 empty tables and restarted it until it hit one of those.

Are there any report this? How come this is the first time I heard of this? How can companies trust this kind of managed DB services?

Post reply on HN