111–120 of 143 posts
> PlanetScale has forked MySQL to add vector storage and search! You’ll be able to support your AI and ML applications with the world’s most scalable database platform. This unifies the reliability and functionality of MySQL with the ability to store vectors and perform similarity search.
Supabase has pgvector extension and that’s enough for my limited RAG use cases. I dont really need to use anything beyond postgres. On the other hand, enterprise might find it easier/cheaper to buy a second db than migrating their existing db to whatever the latest version. I dont think it’s as simple
I still haven't figured what a vector DB is, beyond something something AI.
Imagine: you need a data structure which allows you to store vectors in memory and then say "here is a vector A, give me the 10 vectors which are closest to the same direction of this vector A, in n dimensional space". One can imagine there is some optimal way to lay out the data in memory such that it would be relatively quick to do that. One can imagine a naive way to do it which probably wouldn't be fast. A vector…
This sounds a problem more similar to what 3D physics engines can do, but generalized to higher dimensions, as opposed to traditional text and key-based database stuff.
The algorithms and data structures in physics engines (bounding volume hierarchies, kd-trees, etc.) are quite different from how a traditional database index (B-trees and skip lists are popular there, if I remember correctly) is searched and stored.
Earlier quoted context omitted.
Imagine: you need a data structure which allows you to store vectors in memory and then say "here is a vector A, give me the 10 vectors which are closest to the same direction of this vector A, in n dimensional space". One can imagine there is some optimal way to lay out the data in memory such that it would be relatively quick to do that. One can imagine a naive way to do it which probably wouldn't be fast. A vector…
I think the better question is, what the hell is a vector in the context of normal business logic? Sure, a word embedding vector makes sense because it's all an abstraction anyways, but if I have an "Employee" table with name, address, position, etc columns, how does that translate into a vector?
Here:
https://chat.openai.com/share/9e557a90-e127-4654-9271-7c51fd...
The argument that combining traditional database and vector database into one because it reduces data movement doesn't compute for me. Firstly, even for non vector data, read/write transactional database vs read-optimized store purely for fast serving are already markedly different. Then, the shape of data that is used to generate embeddings is markedly different than the shape of data that is ready to transact or se…
I am super ignorant about vector databases, but don't they store n-dimensional vectors, and aren't they used to answer geometric(ish) problems, getting the k closest entries to a particular point? This sounds a problem more similar to what 3D physics engines can do, but generalized to higher dimensions, as opposed to traditional text and key-based database stuff. The algorithms and data structures in physics engines…
So, by using databases which can efficiently answer geometric nearest-neighbor questions, you can quickly search for chunks of text that are similar to each other.
I am super ignorant about vector databases, but don't they store n-dimensional vectors, and aren't they used to answer geometric(ish) problems, getting the k closest entries to a particular point? This sounds a problem more similar to what 3D physics engines can do, but generalized to higher dimensions, as opposed to traditional text and key-based database stuff. The algorithms and data structures in physics engines…
For example, the text "chocolate milk" is all the same characters as "milk chocolate", but likely have very different usage within the context of retailers or cooks. So, their vectors should be very different.
Word2Vec is NLP that uses a neural net to build these vectors: https://en.wikipedia.org/wiki/Word2vec