I thought you need memory for these things and CPU is not the bottleneck?
I haven’t looked at this repo, but new techniques taking advantage of nvme and io_uring make on disk performance really good without needing to keep everything in RAM.
Zvec: A lightweight, fast, in-process vector database
11–20 of 50 posts
Re: Zvec: A lightweight, fast, in-process vector database
#12Their self-reported benchmarks have them out-performing pinecone by 7x in queries-per-second: https://zvec.org/en/docs/benchmarks/ I'd love to see those results independently verified, and I'd also love a good explanation of how they're getting such great performance.
8K QPS is probably quite trivial on their setup and a 10M dataset. I rarely use comparably small instances & datasets in my benchmarks, but on 100M-1B datasets on a larger dual-socket server, 100K QPS was easily achievable in 2023: https://www.unum.cloud/blog/2023-11-07-scaling-vector-search... ;)
Typically, the recipe is to keep the hot parts of the data structure in SRAM in CPU caches and a lot of SIMD. At the time of those measurements, USearch used ~100 custom kernels for different data types, similarity metrics, and hardware platforms. The upcoming release of the underlying SimSIMD micro-kernels project will push this number beyond 1000. So we should be able to squeeze a lot more performance later this year.
Re: Zvec: A lightweight, fast, in-process vector database
#13How does this compare to duckdbs vector capabilities (vss extension)?
Re: Zvec: A lightweight, fast, in-process vector database
#14Their self-reported benchmarks have them out-performing pinecone by 7x in queries-per-second: https://zvec.org/en/docs/benchmarks/ I'd love to see those results independently verified, and I'd also love a good explanation of how they're getting such great performance.
PGVectorScale claims even more. Also want to see someone verify that.
Re: Zvec: A lightweight, fast, in-process vector database
#15I haven't been following the vector db space closely for a couple years now, but I find it strange that they didn't compare their performance to the newest generation serverless vector dbs: Pinecone Serverless, turbopuffer, Chroma (distributed, not the original single-node implementation). I understand that those are (mostly) hosted products so there's not a true apples-to-apples comparison with the same hardware, but surely the most interesting numbers are cost vs performance.
Re: Zvec: A lightweight, fast, in-process vector database
#16How does this compare to duckdbs vector capabilities (vss extension)?
Yes, nothing on that or sqlite-vec (both of which seem to be apples to apples comparisons).
Re: Zvec: A lightweight, fast, in-process vector database
#17How does this compare to duckdbs vector capabilities (vss extension)?
Yes, nothing on that or sqlite-vec (both of which seem to be apples to apples comparisons). https://zvec.org/en/docs/benchmarks/
I maintain a fork of sqlite-vec (because there hasn't been activity on the main repo for more than a year): sqlite-vec is great for smaller dimensionality or smaller cardinality datasets, but know that it's brute-force, and query latency scales exactly linearly. You only avoid full table scans if you add filterable columns to your vec0 table and include them in your WHERE clause. There's no probabilistic lookup algorithm in sqlite-vec.
Re: Zvec: A lightweight, fast, in-process vector database
#18[dead]
Re: Zvec: A lightweight, fast, in-process vector database
#19[dead]
useful for adding semantic search to tiny bits of data, e.g. collections of research papers in a folder on my computer, etc.
for web stuff, e.g. community/forums/docs/small sites which usually don't even have 1M rows of data, precomputing embeddings and storing them and running on a small vector search like this somewhere is much simpler/cheaper than running external services
it's the operational hassle of not having to deal with a dozen+ external services, logins, apis, even if they're free
(I do like mixed bread for that, but I'd prefer it to be on my own lightweight server or serverless deployment)
Re: Zvec: A lightweight, fast, in-process vector database
#20[deleted]