I had only heard positive things about pgvector but when you Google comparisons with leading vector dbs you keep getting seo slop from Tiger Data pushing pgvector with very suspicious benchmarks that turned me off it altogether instead
https://www.tigerdata.com/blog/pgvector-vs-qdrant
The Case Against PGVector
111–120 of 144 posts
Re: The Case Against PGVector
#112Earlier quoted context omitted.
why is this amazing, it’s just a 1 bit lossy compression representation of the original information? If you have a vector in n-dimensional space this is effectively just representing the basis vectors that the original has.
You can take 8192 bytes of information (1024 x 32 bit floats) and reduce that to 128 bytes (1024 bits, a 64x reduction in size!) and still get results that are about 95% as good. I find that cool and surprising.
1024 bits for a hash is pretty roomy. The embedding "just" has to be well-distributed across enough of the dimensions.
Re: The Case Against PGVector
#113Id love to read a blog post like this about S3 Vector buckets. Does anyone have experience with it in production?
The service is still in preview, so AWS are explicitly telling people not to put it into production. From my non-production experiments with it, the main limitation is that you can only retrieve up to 30 top_k results, which means you can't use it with a re-ranker, or at least not as effectively. For many production use cases that will be a deal breaker.
My issue with it is that it requires a lot of duplication between it and a traditional rdbms; you can’t use it alone because it doesn’t offer filtering without a search vector (i.e. what some vendors call a scroll function).
Re: The Case Against PGVector
#114Earlier quoted context omitted.
In Discourse embeddings power: - Related Topics, a list of topics to read next, which uses embeddings of the current topic as the key to search for similar ones - Suggesting tags and categories when composing a new topic - Augmented search - RAG for uploaded files
what does the rag for uploaded files do in discourse? also, when i run a discourse search does it really do both a regular keyword search and a vector search? how do you combine results? does all discourse instances have those features? for example, internals.rust-lang.org, do they use pgvector?
> what does the rag for uploaded files do in discourse?
You can upload files that will act as RAG files for an AI bot. The bot can also have access to forum content, plus the ability to run tools in our sandboxed JS environment, making it possible for Discourse to host AI bots.
> also, when i run a discourse search does it really do both a regular keyword search and a vector search? how do you combine results?
Yes, it does both. In the full page search it does keyword first, then vector asynchronously, which can be toggled by the user in the UI. It's auto toggled when keyword has zero results now. Results are combined using reciprocal rank fusion.
In the quick header search we simply append vector search to keyword search results when keyword returns less than 4 results.
> does all discourse instances have those features? for example, internals.rust-lang.org, do they use pgvector?
Yes, all use PGvector. In our hosting all instances default to having the vector features enabled, we run embeddings using https://github.com/huggingface/text-embeddings-inference
Re: The Case Against PGVector
#115> None of the blogs mention that building an HNSW index on a few million vectors can consume 10+ GB of RAM or more
Speaking of "production" -- in what world is "10+ GB" a lot of RAM for a database server?
I have to agree: the author should definitely not use Postgres or pgvector in production...
Re: The Case Against PGVector
#116Earlier quoted context omitted.
pgvectorscale is 100% open source please ask your RDS rep to support it we (tiger data) are also happy to help push that along if we can help
Is this something that can happen? We just ran into this limitation and I really want to keep using pgvectorscale... am exploring other solutions on EKS but RDS would be so much easier. From my reading it seems like this isn't something we can get done as a single AWS customer though.
It is up to RDS. But there should be nothing stopping them. AFAIK they respond to customer interest.
Re: The Case Against PGVector
#117'Nobody’s actually run this in production' - the majority of people who work with postgres don't talk about it or gloat about it because it's a tool that works - including it's addons.
Yes, young engineers get all hot and bothered over the most recent tools but - they have no idea how things work and run.
I worked on a project that wanted to use a hot and frothy vector database. The issue - ok, where are we getting the 1/4-1/2 time person to manage it? Product engineers - derp? what? People who live in node and python cutting edge don't really think about the actual production implications of their choices.
Re: The Case Against PGVector
#118Earlier quoted context omitted.
You can take 8192 bytes of information (1024 x 32 bit floats) and reduce that to 128 bytes (1024 bits, a 64x reduction in size!) and still get results that are about 95% as good. I find that cool and surprising.
1024 bits for a hash is pretty roomy. The embedding "just" has to be well-distributed across enough of the dimensions.
Yeah, that's what I was thinking: Did we think 32 bits across each of the 1024 dimensions would be necessary? Maybe 32768 bits is adding unnecessary precision to what is ~1024 bits of information in the first place.
Re: The Case Against PGVector
#119Earlier quoted context omitted.
I think the tricky thing here is that the specific things I referred to (real time writes and pushing SQL predicates into your similarity search) work fine at small scale in such a way that you might not actually notice that they're going to stop working at scale. When you have 100,000 vectors, you can write these SQL predicates (return the 5 top hits where category = x and feature = y) and they'll work fine up until…
> When you have 100,000 vectors [...] and they'll work fine So 95% of use-cases.
In that case you might not even really need optimized vector search though.
Re: The Case Against PGVector
#120Earlier quoted context omitted.
The post is a clear example of when YAGNI backfires, because you think YAGNI but then, you actually do need it. I had this experience, the author had this experience, you might as well - the things you think you AGN are actually pretty basic expectations and not luxuries: being able to write vectors real-time without having to run other processes out of band to keep the recall from degrading over time, being able to…
Many of the concerns in the article could be addressed by standing up a separate PG database that's used exclusively for vector ops and then not using it for your relational data. Then your vector use cases get served from your vector DB and your relational use cases get served from your relational DB. Separating concerns like that doesn't solve the underlying concern but it limits the blast radius so you can operate…
That is a workaround and precisely the point the author makes. It increases operational complexity and creates a divide between records in the vector DB and the relational DB.