11–20 of 26 posts
From a discussion with a friend today.. Are embeddings a hack? Is building out tooling and databases and APIs and companies around embeddings all going to be for naught as soon as there's a solid LLM/API with a big enough context window?
From a discussion with a friend today.. Are embeddings a hack? Is building out tooling and databases and APIs and companies around embeddings all going to be for naught as soon as there's a solid LLM/API with a big enough context window?
The short answer is that, yes, embeddings are probably a hack in the same way that using bits or short variable names were hacks to reduce memory usage. At some point you are correct: someone would prompt "given , answer ".
From a discussion with a friend today.. Are embeddings a hack? Is building out tooling and databases and APIs and companies around embeddings all going to be for naught as soon as there's a solid LLM/API with a big enough context window?
The short answer is that, yes, embeddings are probably a hack in the same way that using bits or short variable names were hacks to reduce memory usage. At some point you are correct: someone would prompt "given , answer ".
From a discussion with a friend today.. Are embeddings a hack? Is building out tooling and databases and APIs and companies around embeddings all going to be for naught as soon as there's a solid LLM/API with a big enough context window?
Earlier quoted context omitted.
The short answer is that, yes, embeddings are probably a hack in the same way that using bits or short variable names were hacks to reduce memory usage. At some point you are correct: someone would prompt "given , answer ".
It's more than just optimizing for space (which is still going to be important), it's also about using vector databases to seed the data from a wider dataset and translating that into something the AI can use. I mean technically in the far future you could dump a whole database into the 'context' and work off of it, but Vector DBs will fill that role in the meantime and add a memory layer on top of it for future quer…
LLMs should not be trained to simply memorize information. Instead, they should be designed to understand and identify patterns in the data, and use the knowledge stored in vector databases to organize and summarize information.
Vector databases can be used to store and organize knowledge in a way that is more accessible to LLMs. By using vector representations, LLMs can easily access and manipulate knowledge, allowing them to more effectively process and analyze large amounts of information.
From a discussion with a friend today.. Are embeddings a hack? Is building out tooling and databases and APIs and companies around embeddings all going to be for naught as soon as there's a solid LLM/API with a big enough context window?
Even with an incredibly long context window (say, 1M tokens), attention still suffers from a problem with long-term dependencies. This is probably why OpenAI hasn't publicly released their 32k token length model just yet.
Earlier quoted context omitted.
The short answer is that, yes, embeddings are probably a hack in the same way that using bits or short variable names were hacks to reduce memory usage. At some point you are correct: someone would prompt "given , answer ".
But embedding-based semantic search can handle arbitrary sized databases. I fully believe context windows are going to grow: I am skeptical they will grow to cover "all your company's documents" or even "the full encyclopedia" sizes.
This is the type of statement that I feel like is often/usually wrong -- at least for the common case. The last time I had this argument was about CDs and how eventually we'll start burning them because they'll be in the cloud, and my friend arguing that storage and network bandwidth would make that impractical if everyone did it.
I expect context window compression or smart ways to embed them so they still provide useful context in "most" cases, even if not-lossless, will be an active area of research.
EDIT: That said, looking a the original question -- I do think vector embeddings are still useful in their own right and somewhat orthogonal to context window sizes. IMO.
Earlier quoted context omitted.
Even with an incredibly long context window (say, 1M tokens), attention still suffers from a problem with long-term dependencies. This is probably why OpenAI hasn't publicly released their 32k token length model just yet.
I think they haven't released it because the capabilities it has are simply too powerful when combined with a vectorDB.