51–60 of 66 posts
Proprietary closed-source lock-in. Nothing to see here.
Seriously. The amount of lift a SaaS product needs to give me is insane for me to even bother evaluating it, and there's a near zero percent chance I'll use it in my core.
Earlier quoted context omitted.
Show your math lol
I assume by "node" OP meant something like a DGX node. Which yea, that would work, but not everyone (no one?) wants to buy a 500k system to do vector search. B200 spec: * 8TB/sec HBM bandwidth * 10 PetaOPs assuming int8. * 186GB of VRAM. If we work with 512-dimensional int8 embeddings, then we need 512GB VRAM to hold them, so assuming we have 8xB200 node (~500k$++), we can easily hold them (125M vectors per GPU). It…
It's crazy how people add bloat and complexity to their stuff just because they want to do medium scale RAG with ca. 2 million embeddings.
Here comes the punchline, you do not need a fancy vector database in this case. I stumbled over https://github.com/sqliteai/sqlite-vector which is a SQLite extension and I wonder why no one else did this before, but it simply implements a highly optimized brute force search over the vectors, so you get sub 100ms queries over millions of vectors with perfect recall. It uses dynamic runtime dispatch that makes use of the available SIMD instructions your CPU has. Turns out this might be all you need. No need for memory a memory hungry search index (like HNSW) or writing a huge index to disk (like DiskANN).
Earlier quoted context omitted.
I’m curious, what’s the tech stack behind this?
Vectroid is pure Java solution based on modified version of Lucene. We use a custom built FileSystem to work directly with GCS (Google cloud object store). It is a terraform/helm managed Kubernetes deployment.
I think the whole field of vector databases is mostly just one huge misunderstanding. Most of you are not Google or any other big tech company so so won't have billions of embeddings. It's crazy how people add bloat and complexity to their stuff just because they want to do medium scale RAG with ca. 2 million embeddings. Here comes the punchline, you do not need a fancy vector database in this case. I stumbled over h…
> For production or managed service use, please contact SQLite Cloud, Inc for a commercial license.
How is this different from running tuned HNSW vector indices on Elasticsearch?
How is this different from running tuned HNSW vector indices on Elasticsearch?
Lucene is tough to deal with. About 15 hours ago — right when this comment was posted — I was giving a talk at Databricks comparing the world’s most widely used search engines. I’ve never run into as many issues with any other similar tool as I did with Lucene. To be fair, it’s been around for ~26 years and has aged remarkably well... but it’s the last thing I’d choose today.
Elasticsearch is at least good / at hiding the Lucene zoo under the hood.
I think the whole field of vector databases is mostly just one huge misunderstanding. Most of you are not Google or any other big tech company so so won't have billions of embeddings. It's crazy how people add bloat and complexity to their stuff just because they want to do medium scale RAG with ca. 2 million embeddings. Here comes the punchline, you do not need a fancy vector database in this case. I stumbled over h…
Might be all you need, except an open source licence: > For production or managed service use, please contact SQLite Cloud, Inc for a commercial license.
How is this different from running tuned HNSW vector indices on Elasticsearch?
Lucene is tough to deal with. About 15 hours ago — right when this comment was posted — I was giving a talk at Databricks comparing the world’s most widely used search engines. I’ve never run into as many issues with any other similar tool as I did with Lucene. To be fair, it’s been around for ~26 years and has aged remarkably well... but it’s the last thing I’d choose today.
I went looking around last year and couldn’t really find many options, but I might have been looking in the wrong places.
I think the whole field of vector databases is mostly just one huge misunderstanding. Most of you are not Google or any other big tech company so so won't have billions of embeddings. It's crazy how people add bloat and complexity to their stuff just because they want to do medium scale RAG with ca. 2 million embeddings. Here comes the punchline, you do not need a fancy vector database in this case. I stumbled over h…
https://github.com/duckdb/duckdb-vss
Since duckdb is already columnar, it goes brrrrr with single digit millisecond vector similarly lookups.