Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Every database will become a vector database sooner or later

nextword.substack.com

Post reply on HN
Print view View original

121–130 of 143 posts

Re: Every database will become a vector database sooner or later

#121

Post by fhaltmayer »

jbird11 wrote: post #76

Lots of focus on RAG here, and rightfully so, but I feel an overlooked benefit to vector databases is the novel visualizations they provide. To be able to plot qualitative data onto a 2d graph with tsne reduction provides a new method with which to draw insights from. I think many companies would benefit from such a visualization tool, especially those in qualitative research.

Unfortunately that visualization doesn't really work with vector dbs. Vector dbs normally split their data into separate segments and build indexes on them separately. There is no one overarching index but rather many small ones that are searched in parallel. In addition to this, such a high compression to reduce it down to 2d/3d ends up becoming a giant blob without too much information.

Re: Every database will become a vector database sooner or later

#122

Post by BoorishBears »

weird-eye-issue wrote: post #94
BoorishBears wrote: post #60

Earlier quoted context omitted.

Bloat your DB... or pull in an entirely new vendor and bloat your entire operational outlay. I'd really love to know what kind of insane scale justifies that tradeoff...

That's a very exaggerated way to look at things lol. Nothing got bloated at all in this process, we are just using the right tools for the job. I'm a solo founder and the only backend developer. I can assure you this decision only made my life easier by choosing the correct tech from the get go.

Nothing exaggerated, your comment implied scale was your justification: in which case there'd better have been some crazy high load that just brought the tool you already had to its knees to justify paying an additional closed source platform and manually having to pipe data to it in addition to your main data store.

Of course if I sounded incredulous it's because I didn't think you had that scale, and it sounds like I was correct?

Re: Every database will become a vector database sooner or later

#123

Post by m00x »

vinay_ys wrote: post #91

The argument that combining traditional database and vector database into one because it reduces data movement doesn't compute for me. Firstly, even for non vector data, read/write transactional database vs read-optimized store purely for fast serving are already markedly different. Then, the shape of data that is used to generate embeddings is markedly different than the shape of data that is ready to transact or se…

> So, no matter where it is stored, it has to leave that store, get transformed and enriched and then run through an embeddings generator (ML inference).

Same for pretty much all the data in your database. It comes from an app, that does validation/editing/transforms. The benefit is that it's all together, can be atomic, only requires one query to get, update, and delete.

Vector databases are just normal databases with a vector index. There's no reason for you to have a specialized DB for it.

Also, embeddings aren't inference, it's a token lookup. There's no forward pass.

Re: Every database will become a vector database sooner or later

#124

Post by smilliken »

sgarland wrote: post #80
deepGem wrote: post #73

Earlier quoted context omitted.

How is indexing a vector different from indexing a varchar or an integer ? If you convert a vector into a byteaarray it should be no different from a bytearray of varchar but for the bytearray contents. Now if you want to do similarity search you have to measure the distance between 2 or more vectors and that's independent of the indexing. No ? So any database with sufficient memory should be able to accomplish this…

Mostly the number of dimensions. Assuming your vectors are float16, so 2 bytes each, you’d run into Postgres’ B+tree index limit (2704 bytes) very quickly. You could index a 512-dimension vector fine, but I believe most models are well beyond that. There are alternative index types, of course, or you could index the hash of the vector. These both come with tradeoffs.

Btree isn't a very useful index type for a vector, though. GIN, GIST, and the handful of new extensions optimizing for vector search are what you'd want (and don't have this limitation).

Aside, you can increase the size of tuples you can index in a postgresql btree by increasing the postgresql page size (requires a recompile and creating a new database instance).

Re: Every database will become a vector database sooner or later

#125

Post by redwood »

gk1 wrote: post #69

Go ahead and try to load a billion embeddings into Mongo or Elastic. The author says this will be "faster, cheaper, and simpler..." Will it? (Disclaimer: I work for Pinecone, so obvious bias ahead but also perspective of 3 years since launching the Vector DB category and actually seeing billion-scale vector search deployments.) > Basically, having separate vector DBs can add to cost and complexity. Imagine you were a…

I am curious, do you see a lot of transactional / high velocity updates to vector embeddings in an underlying operational database system? Guess we'll see more of that in future and still at the beginning? I ask b/c you describe regularly writing the data and latency to merge/keep up BUT one could argue that using a completely distinct system makes it even further from the source of truth and hence inherently harder to keep up

Re: Every database will become a vector database sooner or later

#126

Post by gk1 »

redwood wrote: post #125
gk1 wrote: post #69

Go ahead and try to load a billion embeddings into Mongo or Elastic. The author says this will be "faster, cheaper, and simpler..." Will it? (Disclaimer: I work for Pinecone, so obvious bias ahead but also perspective of 3 years since launching the Vector DB category and actually seeing billion-scale vector search deployments.) > Basically, having separate vector DBs can add to cost and complexity. Imagine you were a…

I am curious, do you see a lot of transactional / high velocity updates to vector embeddings in an underlying operational database system? Guess we'll see more of that in future and still at the beginning? I ask b/c you describe regularly writing the data and latency to merge/keep up BUT one could argue that using a completely distinct system makes it even further from the source of truth and hence inherently harder…

Yes, for example any site with user-uploaded content — think marketplaces, social media, SaaS file/docs storage, etc. Very high write throughput and expectation of that data being available in search/chatbots right away.

Re: Every database will become a vector database sooner or later

#127

Post by gk1 »

tinyhouse wrote: post #102
gk1 wrote: post #69

Go ahead and try to load a billion embeddings into Mongo or Elastic. The author says this will be "faster, cheaper, and simpler..." Will it? (Disclaimer: I work for Pinecone, so obvious bias ahead but also perspective of 3 years since launching the Vector DB category and actually seeing billion-scale vector search deployments.) > Basically, having separate vector DBs can add to cost and complexity. Imagine you were a…

Your argument is reasonable, however, you haven't explained why a purpose-built vector database is more scalable than any other DB with vector search support. Why is Elastic much slower than Pinecone? Is it because Pinecone is optimized in ways that Elastic cannot be? Is it because the Pinecone team has much better understanding on how to optimize vector search algorithms? or is it something else?

Blog post coming soon about this. The TL;DR is the underlying architecture matters… a lot. More coming soon.

Re: Every database will become a vector database sooner or later

#128

Post by sgarland »

smilliken wrote: post #124
sgarland wrote: post #80

Earlier quoted context omitted.

Mostly the number of dimensions. Assuming your vectors are float16, so 2 bytes each, you’d run into Postgres’ B+tree index limit (2704 bytes) very quickly. You could index a 512-dimension vector fine, but I believe most models are well beyond that. There are alternative index types, of course, or you could index the hash of the vector. These both come with tradeoffs.

Btree isn't a very useful index type for a vector, though. GIN, GIST, and the handful of new extensions optimizing for vector search are what you'd want (and don't have this limitation). Aside, you can increase the size of tuples you can index in a postgresql btree by increasing the postgresql page size (requires a recompile and creating a new database instance).

Agreed to the first, but you have to first know that those exist (and what they’re good for). This leads into my second point: IME, the Venn diagram for “people making AI stuff” and “people capable of compiling and running their own DB in a reliable manner” has no overlap.

Re: Every database will become a vector database sooner or later

#129

Post by redwood »

gk1 wrote: post #126
redwood wrote: post #125

Earlier quoted context omitted.

I am curious, do you see a lot of transactional / high velocity updates to vector embeddings in an underlying operational database system? Guess we'll see more of that in future and still at the beginning? I ask b/c you describe regularly writing the data and latency to merge/keep up BUT one could argue that using a completely distinct system makes it even further from the source of truth and hence inherently harder…

Yes, for example any site with user-uploaded content — think marketplaces, social media, SaaS file/docs storage, etc. Very high write throughput and expectation of that data being available in search/chatbots right away.

But in a sense doesn't that lend credence to the idea that the closer the indexing is happening to where the data is born, there may be some natural advantages?

Re: Every database will become a vector database sooner or later

#130

Post by jbird11 »

fhaltmayer wrote: post #121
jbird11 wrote: post #76

Lots of focus on RAG here, and rightfully so, but I feel an overlooked benefit to vector databases is the novel visualizations they provide. To be able to plot qualitative data onto a 2d graph with tsne reduction provides a new method with which to draw insights from. I think many companies would benefit from such a visualization tool, especially those in qualitative research.

Unfortunately that visualization doesn't really work with vector dbs. Vector dbs normally split their data into separate segments and build indexes on them separately. There is no one overarching index but rather many small ones that are searched in parallel. In addition to this, such a high compression to reduce it down to 2d/3d ends up becoming a giant blob without too much information.

Maybe I'm misunderstanding what exactly a vector db is.

Let's say you have a chatbot and stored in its database is the usual info like session id, timestamp, message etc. To "vectorize" this db then would be to vectorize all the messages. Is this too simple an understanding?

Once the db has been vectorized then we can do semantic search on the messages and create more informative graphs based off the semantic similarity for messages within a given timeframe or other criteria.

Post reply on HN