I’m using sqlite-vec along with FTS5 in (you guessed it) SQLite and it’s pretty cool. :)
[deleted]
Vector databases are the wrong abstraction
51–60 of 95 posts
Re: Vector databases are the wrong abstraction
#52I’m using sqlite-vec along with FTS5 in (you guessed it) SQLite and it’s pretty cool. :)
what's your experience with sqlite-vec? I'm considering using sqlite-vec in addition to/or replace qdrant vector db for a project (recurse.chat), since I'm moving all the data to sqlite. I love everything SQLite so far, but haven't got to try out sqlite-vec yet.
Re: Vector databases are the wrong abstraction
#53Or you can use Cassandra / DataStax Astra to store the original text, the metadata and the embedding in a single table and then do hybrid queries against them (with pre- or post-filtering, optimized automatically).
Re: Vector databases are the wrong abstraction
#54Clever!
A method that has worked well for me: divorced databases.
The first database is a plaintext database that stores rows: id, data, and metadata and the second database is a vector database that stores id, embedding. whenever a new row is added the first database makes a POST request to the second database. The second database embeds the data and returns the id of its row. The first database uses that ID to store the plain text.
When searching, the second database is optimized for cosine sim with an HNSW index. It returns the IDs to the first database, which fetch the plaintext to return to the user.
The advantages of this are that the plaintext data can be A/B tested across multiple embedding models without affecting the source, and each database can be provisioned for a specific task. Also lowers hosting costs and security because there only needs to be one central vector database and small provisioned plaintext databases.
Re: Vector databases are the wrong abstraction
#55This reads solely as a sales pitch, which quickly cuts to the "we're selling this product so you don't have to think about it." ...when you actually do want to think about it (in 2024). Right now, we're collectively still figuring out: 1. Best chunking strategies for documents 2. Best ways to add context around chunks of documents 3. How to mix and match similarity search with hybrid search 4. Best way to version and…
Points 2-4 are clear pointers to a real database as the home for vector data & search.
Re: Vector databases are the wrong abstraction
#56Earlier quoted context omitted.
as far as I can tell Chroma can only store chunks, not the original documents. This is from your docs `If the documents are too large to embed using the chosen embedding function, an exception will be raised`. In addition it seems that embeddings happen at ingest time. So, if, for example, the OpenAI endpoint is down the insert will fail. That, in turn means your users need to use a retry mechanism and a queuing syst…
Chroma certainly doesn't have the most advanced API in this area, but you can for sure store chunks or documents, its up to you. If your document size is too large to generate embeddings in a single forward pass, then yes you do need to chunk in that scenario. Oftentimes though, even if the document does fit, you choose to chunk anyways or further transform the data with abstractive/extractive summarization technique…
(Post co author) We absolutely agree that chunking is critical for good RAG. What I think you missed in our post is that the vectorizer allows you to configure a chunking strategy of your choice. So you store the full doc but then the system well chunk and embed it for you. We don’t blindly embed the full document.
Re: Vector databases are the wrong abstraction
#57Great point! (Disclaimer: I work for Elastic) Elasticsearch has recently added a data type called semantic_text, which automatically chunks text, calculates embeddings, and stores the chunks with sensible defaults. Queries are similarly simplified, where vectors are calculated and compared internally, which makes a lot less I/O and a lot simpler client code. https://www.elastic.co/search-labs/blog/semantic-search-sim…
I made something similar, but used duckDB as the vector store (and query engine)! It’s impressively fast https://github.com/patricktrainer/duckdb-embedding-search
Amy specific reason to use dDB?
I've got a crapload of json q & a formatted discussions on a topic, and am trying to figure out if I just store it somewhere and query it, or do I also do vector embeddings, kinda lost with all the possible options.
Re: Vector databases are the wrong abstraction
#58I agree that putting the vectors in a separate DB often does not makes. Just use Hana https://news.sap.com/2024/04/sap-hana-cloud-vector-engine-ai... ;-) IMHO putting the calculation of the embedding vectors into the db (even if it is just a remote call) is not a got idea. How do you react to failures of the remote call, security issues because of code running within your DB ..?
Re: Vector databases are the wrong abstraction
#59Hey HN! Post co-author here, excited to share our new open-source PostgreSQL tool that re-imagines vector embeddings as database indexes. It's not literally an index but it functions like one to update embeddings as source data gets added, deleted or changed. Right now the system only supports OpenAI as an embedding provider, but we plan to extend with local and OSS model support soon. Eager to hear your feedback and…
Hey, this is really cool! Thanks for the article and the tool itself.
One question - in the RAG projects we've done, most of the source data was scattered in various source systems, but wasn't necessarily imported into a single DB or Data Lake. For example, building an internal Q&A tool for a company that has knowledge stored in services like Zendesk, Google Drive, an internal company Wiki, etc.
In those cases, it made sense to not import the source documents, or only import metadata about them, and keep the embeddings in a dedicated Vector DB. This seems to me to be a fairly common use case - most enterprises have this kind of data scattered across various systems.
How do you envision this kind of use case working with this tool? I may have missed it, but you mention things like working with images, etc, is your assumption that everyone is storing all of that data in Postgres?
Re: Vector databases are the wrong abstraction
#60In the project I'm currently working on, I use OpenSearch for RAG because it allows me to use hybrid search which combines full-text search with vector search, and OpenSearch does all the math combining two result sets for me. Research shows that hybrid search can give better results than just vector search alone. Another team was already integrating OpenSearch for full text search for a different feature, so I just reused exising infra, sparing the time of DevOps/SRE.