Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Case Against PGVector

alex-jacobs.com

Post reply on HN
Print view View original

101–110 of 144 posts

Re: The Case Against PGVector

#101

Post by whakim »

xfalcox wrote: post #100
whakim wrote: post #97

Earlier quoted context omitted.

Interested to hear more about your experience here. At Halcyon, we have trillions of embeddings and found Postgres to be unsuitable at several orders of magnitude less than we currently have. On the iterative scan side, how do you prevent this from becoming too computationally intensive with a restrictive pre-filter, or simply not working at all? We use Vespa, which means effectively doing a map-reduce across all of…

We host thousands of forums but each one has its own database, which means we get a sort of free sharding of the data where each instance has less than a million topics on average. I can totally see that at a trillion scale for a single shard you want a specialized dedicated service, but that is also true for most things in tech when you get to the extreme scale .

Thanks for the reply! This makes much more sense now. To preface, I think pgvector is incredibly awesome software, and I have to give huge kudos to the folks working on it. Super cool. That being said, I do think the author isn't being unreasonable in that the limitations of pgvector are very real when you're talking indices that grow beyond millions of things, and the "just use pgvector" crowd in general doesn't have a lot of experience with scaling things beyond toy examples. Folks should take a hard look at what size they expect their indices to grow to in the near-to-medium-term future.

Re: The Case Against PGVector

#102

Post by ezekiel68 »

> Your database is now handling your normal transactional workload, analytical queries, AND maintaining graph structures in memory for vector search.

No. No one in production is trying to use the same instance for all of these use-cases at scale. The fundamental misunderstanding here is assuming or even "demanding" that one instance should be able to provide OLTP, OLAP and vector ops with no compromises. The workloads are fundamentally different and doing serious work requires architecting the solution much more intelligently.

Re: The Case Against PGVector

#103

Post by nextaccountic »

xfalcox wrote: post #51
dpflan wrote: post #33

Earlier quoted context omitted.

What are you using it for? Is it part of a hybrid search system (keyword + vector)?

In Discourse embeddings power: - Related Topics, a list of topics to read next, which uses embeddings of the current topic as the key to search for similar ones - Suggesting tags and categories when composing a new topic - Augmented search - RAG for uploaded files

what does the rag for uploaded files do in discourse?

also, when i run a discourse search does it really do both a regular keyword search and a vector search? how do you combine results?

does all discourse instances have those features? for example, internals.rust-lang.org, do they use pgvector?

Re: The Case Against PGVector

#104

Post by tveita »

simonw wrote: post #72
xfalcox wrote: post #18

Earlier quoted context omitted.

Also worth mentioning that we use quantization extensively: - halfvec (16bit float) for storage - bit (binary vectors) for indexes Which makes the storage cost and on-going performance good enough that we could enable this in all our hosting.

It still amazes me that the binary trick works. For anyone who hasn't seen it yet: it turns out many embedding vectors of e.g. 1024 floating point numbers can be reduced to a single bit per value that records if it's higher or lower than 0... and in this reduced form much of the embedding math still works! This means you can e.g. filter to the top 100 using extremely memory efficient and fast bit vectors, then run a…

Depending on your data you might also get better results by applying a random rotation to your vector before quantization.

https://ieeexplore.ieee.org/abstract/document/6296665/ (https://refbase.cvc.uab.cat/files/GLG2012b.pdf)

Re: The Case Against PGVector

#105

Post by mfrye0 »

xfalcox wrote: post #18
xfalcox wrote: post #3

> Nobody’s actually run this in production We do at Discourse, in thousands of databases, and it's leveraged in most of the billions of page views we serve. > Pre- vs. Post-Filtering (or: why you need to become a query planner expert) This was fixed in version 0.8.0 via Iterative Scans ( https://github.com/pgvector/pgvector?tab=readme-ov-file#iter... ) > Just use a real vector database If you are running a single ser…

Also worth mentioning that we use quantization extensively: - halfvec (16bit float) for storage - bit (binary vectors) for indexes Which makes the storage cost and on-going performance good enough that we could enable this in all our hosting.

I was going to say the same. We're using binary vectors in prod as well. Makes a huge difference in the indexes. This wasn't mentioned once in the article.

Re: The Case Against PGVector

#106

Post by muzani »

"Turbopuffer starts at $64 month with generous limits."

Yup, I think this here explains the popularity of pgvector. If $64/month seems like a lot to you, just use pgvector. If it seems cheap, then your usage is complex enough to want a proper vector DB.

Re: The Case Against PGVector

#107

Post by 3abiton »

simonw wrote: post #72
xfalcox wrote: post #18

Earlier quoted context omitted.

Also worth mentioning that we use quantization extensively: - halfvec (16bit float) for storage - bit (binary vectors) for indexes Which makes the storage cost and on-going performance good enough that we could enable this in all our hosting.

It still amazes me that the binary trick works. For anyone who hasn't seen it yet: it turns out many embedding vectors of e.g. 1024 floating point numbers can be reduced to a single bit per value that records if it's higher or lower than 0... and in this reduced form much of the embedding math still works! This means you can e.g. filter to the top 100 using extremely memory efficient and fast bit vectors, then run a…

Now that you mention that, I wonder if LSH would perform better with slightly higher memory footprint

Re: The Case Against PGVector

#108

Post by codingjaguar »

This quite aligns with our observation at Milvus. Recently, we helped several users migrate from pgvector as the workload grew substantially.

It’s worth recognising the strengths of pgvector:

• For small-to-medium scale workloads (e.g., up to millions of vectors, relatively static data), embedding storage and similarity queries inside Postgres can be a simple, familiar architecture.

• If you already use Postgres and your vector workloads are light (low QPS, few dimensions, little metadata filtering / low concurrency), then piggy-backing vector search on Postgres is attractive: minimal added infrastructure.

• For teams that don’t want to introduce a separate vector service, or want to keep things within an existing RDBMS, pgvector is a compelling choice.

From our experience helping users scale vector search in production, several pain-points emerge when scaling vector workloads inside a general-purpose RDBMS like Postgres:

1. Index build / update overhead • Postgres isn’t built from the ground-up for high-velocity vector insertions plus large-scale approximate nearest neighbour (ANN) index maintenance, for example, lacking RaBitQ binary quantization supported in purpose built vector db like Milvus.

• For large datasets (tens/hundreds of millions or beyond), building or rebuilding HNSW/IVF indices inside Postgres can be memory- and time-intensive.

• In production systems where vectors are continuously ingested, updated, deleted, this becomes operationally tricky.

2. Filtered search

• Many use-cases require combining vector similarity with scalar/metadata filters (e.g., “give me top 10 similar embeddings where user_status = ‘active’ AND time > X”).

• Need to understand low level planner to juggle pre-filtering, post-filtering, and planner’s cost model wasn’t built for vector similarity search. For a system not designed primarily as a vector DB, this gets complex. Users shouldn't have to worry about such low level details.

3. Lack of support for full-text search / hybrid search

• Purpose built vector db such as Milvus has mature full-text search / BM25 / Sparse vector support.

Re: The Case Against PGVector

#109

Post by jrochkind1 »

> What bothers me most: the majority of content about pgvector reads like it was written by someone who spun up a local Postgres instance, inserted 10,000 vectors, ran a few queries, and called it a day.

this is a big problem in programmer blog posts. It used to be I could find blog posts by peopel who had actually done the thing ("in anger").

Now it's someone who decided writing up the thing would draw clicks, and googled just enough to write the thing, may or may not have actually even fired it up at all -- may not have even written it, perhaps had AI write it.

It makes any of these blog posts pretty terrible guides.

I used to try at least downvoting these on say reddit when it was obviously not written by someone who had their own actual earned knowledge about the thing, but just gave up, because it's nearly everything.

Re: The Case Against PGVector

#110

Post by remich »

akulkarni wrote: post #59
tacoooooooo wrote: post #27

Earlier quoted context omitted.

I mention this towards the end of the post. it looks like a good solution, but it's not available on RDS

pgvectorscale is 100% open source please ask your RDS rep to support it we (tiger data) are also happy to help push that along if we can help

Is this something that can happen? We just ran into this limitation and I really want to keep using pgvectorscale... am exploring other solutions on EKS but RDS would be so much easier. From my reading it seems like this isn't something we can get done as a single AWS customer though.
Post reply on HN