121–130 of 144 posts
"Turbopuffer starts at $64 month with generous limits." Yup, I think this here explains the popularity of pgvector. If $64/month seems like a lot to you, just use pgvector. If it seems cheap, then your usage is complex enough to want a proper vector DB.
Earlier quoted context omitted.
how are all of the mentioned issues resolved?
So its a longish article and doing a point by point explanation is probably too much for a single post. But several of the points are solved but just standing up a specific Postgres instance for the vector use cases instead of doing this inside an existing instance. Most of the rest of his complaints comes down to this is complex stuff. True, but its not a solution, its a tool used in making a solution. So when using…
He very clearly complains that IVFFlat indexes have to be periodically rebuilt, that HNSW has high overhead (both during inserts and rebuilds) and that the query planner is not particularly good at optimizing queries involving this kind of indexes. None of this is a problem if the dataset is puny enough, but deadly if you want to scale up without investing significant engineering.
Earlier quoted context omitted.
It still amazes me that the binary trick works. For anyone who hasn't seen it yet: it turns out many embedding vectors of e.g. 1024 floating point numbers can be reduced to a single bit per value that records if it's higher or lower than 0... and in this reduced form much of the embedding math still works! This means you can e.g. filter to the top 100 using extremely memory efficient and fast bit vectors, then run a…
I was taken back when I saw what was basically zero recall loss in the real world task of finding related topics, by doing the same thing you described where we over capture with binary embeddings, and only use the full (or half) precision on the subset. Making the storage cost of the index 32 times smaller is the difference of being able to offer this at scale without worrying too much about the overhead.
By moving the values to a single bit, you’re lumping stuff together that was different before, so I don’t think recall loss would be expected.
Also: even if your vector is only 100-dimensional, there already are 2^100 different bit vectors. That’s over 10^30.
If your dataset isn’t gigantic and has documents that are even moderately dispersed in that space, the likelihood of having many with the same bit vector isn’t large.
Earlier quoted context omitted.
The post is a clear example of when YAGNI backfires, because you think YAGNI but then, you actually do need it. I had this experience, the author had this experience, you might as well - the things you think you AGN are actually pretty basic expectations and not luxuries: being able to write vectors real-time without having to run other processes out of band to keep the recall from degrading over time, being able to…
Many of the concerns in the article could be addressed by standing up a separate PG database that's used exclusively for vector ops and then not using it for your relational data. Then your vector use cases get served from your vector DB and your relational use cases get served from your relational DB. Separating concerns like that doesn't solve the underlying concern but it limits the blast radius so you can operate…
Man, that table comparison definitely looks like it was AI generated. I'm starting to question the whole article itself, now :/
> It’s a compelling story. And like most of the AI influencer bullshit that fills my timeline, it glosses over the inconvenient details.
Earlier quoted context omitted.
I was taken back when I saw what was basically zero recall loss in the real world task of finding related topics, by doing the same thing you described where we over capture with binary embeddings, and only use the full (or half) precision on the subset. Making the storage cost of the index 32 times smaller is the difference of being able to offer this at scale without worrying too much about the overhead.
> I was taken back when I saw what was basically zero recall loss in the real world task of finding related topics By moving the values to a single bit, you’re lumping stuff together that was different before, so I don’t think recall loss would be expected. Also: even if your vector is only 100-dimensional, there already are 2^100 different bit vectors. That’s over 10^30. If your dataset isn’t gigantic and has docume…
Earlier quoted context omitted.
I think the tricky thing here is that the specific things I referred to (real time writes and pushing SQL predicates into your similarity search) work fine at small scale in such a way that you might not actually notice that they're going to stop working at scale. When you have 100,000 vectors, you can write these SQL predicates (return the 5 top hits where category = x and feature = y) and they'll work fine up until…
> When you have 100,000 vectors [...] and they'll work fine So 95% of use-cases.
"Turbopuffer starts at $64 month with generous limits." Yup, I think this here explains the popularity of pgvector. If $64/month seems like a lot to you, just use pgvector. If it seems cheap, then your usage is complex enough to want a proper vector DB.
It is most often not the $64. It is about being in sovereign control of your dataplane.
Yesterday, I had a conversation with someone whether it's better to squeeze it all into the system prompt. Someone else argued that even with large context, it's difficult to fit 10k questions into a system prompt.
So I think there's just this mismatch in usage.