Interesting project. How does it fare on scaling? We were evaluating vector dbs and since we were into b2b saas, we are keen on the sharding, scaling and multi tenancy features. currently we are more inclined towards milvus. https://zilliz.com/comparison/qdrant-vs-milvus
Show HN: Epsilla – Open-source vector database with low query latency
21–26 of 26 posts
Re: Show HN: Epsilla – Open-source vector database with low query latency
#22Vector databases seem to be a dime a dozen, now, as well as being built into Elasticsearch and available as Postgres extensions. As far as I know they’re all relatively undifferentiated in performance and features. Is there a viable long-term business here?
This one has more reason to exist than most vector DBs since it's not just a wrapper around hnswlib.
Well, hnswlib is actually faster than epsilla according to benchmarks (compare their own vs ann benchmark), especially in terms of throughput
Re: Show HN: Epsilla – Open-source vector database with low query latency
#23Earlier quoted context omitted.
This one has more reason to exist than most vector DBs since it's not just a wrapper around hnswlib.
Well, hnswlib is actually faster than epsilla according to benchmarks (compare their own vs ann benchmark), especially in terms of throughput
They claim that they are 10x faster given the high accuracy target ( no clue what that means in practice for the AI use case, probably less tokens for the LLM). Can you elaborate why you think hnswlib is still faster? Can you link the benchmark you mention.
Re: Show HN: Epsilla – Open-source vector database with low query latency
#24Earlier quoted context omitted.
Well, hnswlib is actually faster than epsilla according to benchmarks (compare their own vs ann benchmark), especially in terms of throughput
They claim that they are 10x faster given the high accuracy target ( no clue what that means in practice for the AI use case, probably less tokens for the LLM). Can you elaborate why you think hnswlib is still faster? Can you link the benchmark you mention.
Sure. The benchmark from Epsilla https://miro.medium.com/v2/resize:fit:1400/format:webp/1*dDy..., the benchmark for the same dataset and same K (10) from ann benchmark https://ann-benchmarks.com/gist-960-euclidean_10_euclidean.h...
At fixed recall (which is the mentioned accuracy) 0.95 Epsilla gets 200 QPS using multiple intra-threads and a single inter-thread. Hnswlib gets more than 370 QPS at higher 0.97 recall and both single intra- and inter- threads, which is much faster and uses less CPU.
Because hnswlib does not use intra-threads it will scale much better in terms of full throughput, probably close to 7X-8X with 16 threads on 16 vCPUs (compared to Epsilla which saturates with 2.2X improvement from multiple threads). The main premise of Epsilla's solution is trading throughput for latency, which is probably legit but would not work for all.
Note that even though the hardware between the benchmarks is not controlled (Epsilla only says it is some AWS EC2 16C32G, ann benchmark uses AWS r6i.16xlarge), it does not matter that much since the single threaded cpu speeds are pretty stagnant over the years, so ann benchmark single-thread results can be transferred (unless Epsilla is using non-x64 hardware, which would be a weird choice). There is a constant overhead from communication between the nodes in Epsilla, but it is constant and should not affect the speed at high recalls (for which the hnswlib is also faster).
Re: Show HN: Epsilla – Open-source vector database with low query latency
#25“Hippocampus of AI” in a readme is a yellow flag
Why it's a yellow flag?
Re: Show HN: Epsilla – Open-source vector database with low query latency
#26I'm curious about your approach on where you draw the line for database features; I don't have a perspective on what's right, just trying to get informed. There are a bunch of possible areas to circle or ignore when making an ML-capable database of some sort. In rough order of data complexity: 1. Embeddings (context-free vectors, just an ID and the vector) 2. Metadata + Embedding (source data, JSON) 3. Binary Data +…
Thank you for the insightful topic! By reading the question itself drive me think a lot. For the database perspective, instead of dividing the table schema into 3 parts: id, metadata, embedding, we designed in a way closer to SQL, treat vector as another data type, and let user to define any number of fields in a table. ID is just an annotation of a field (composite key might be overkilling for now). There will be an…
Thanks for the thoughts. I agree that you're not going to disintermediate existing datalakes, no matter how successful, so integration makes sense.
Every few months I run up into a use case where I'm like "I want to get a whole bunch of data, analyze it, then search for it later with embeddings, and probably keep running different sorts of analysis on it, and store the embeddings of those analyses in a related way." This still feels fairly difficult to do, or at least there aren't canonical "right" architectures yet.
My instinct is if you nail the ml+dev+data ops needs with good architecture and api you could really have something -- good luck!