21–30 of 140 posts
Here’s an even simpler take: just embed everything the first time, then track what was changed. Use a cheap model to summarize and clean up the documents/chats with summary and keywords. Unless you have entire libraries of books to embed it’s going to be a few hundred dollars of API calls. Then, throw it all in BigQuery. Handles all the vector stuff natively. Sprinkle an agentic bot UI thing on top to make it appear…
The article sounds like AI slop with some predictable tells like short punctual sentences, bizarre jargon, and titles like "Recipe 4: On-The-Fly Embedding (The Fresh Data Play)" Can we not reward junk like this? Most of the sentences are incomprehensible and provide zero actual argumentation, it's just a list of "whats" with no "whys"
Here’s an even simpler take: just embed everything the first time, then track what was changed. Use a cheap model to summarize and clean up the documents/chats with summary and keywords. Unless you have entire libraries of books to embed it’s going to be a few hundred dollars of API calls. Then, throw it all in BigQuery. Handles all the vector stuff natively. Sprinkle an agentic bot UI thing on top to make it appear…
Yep, lock into some vendor from day 1. Great idea!
Maybe I'm old but where exactly are the "dragons"? How is RAG any different from the search systems we've been building before LLMs? Is it the sudden need for everyone to design a search API and engine that's driven this trend? If so, I'd like to see more design patterns around existing search problems: - Correcting or backtracking based on feedback. - Measuring relevance. - Comparison with task-based pre-written que…
The whole embedding thing which converts “tokens” to vectors, which you then store in a vector database so that you can later query by vector distance, seems to be LLM specific technology, no? As far as I know the vectors look a lot like the weights in a LLM itself which is why the vector search also works with some level of intelligence.
at least to me that seems the same as https://en.wikipedia.org/wiki/Word2vec for e.g.
I have a particular antipathy for articles too lazy to spell out acronyms on first use. So: https://en.wikipedia.org/wiki/Retrieval-augmented_generation
OT but its interesting that none of the harnesses today use embeddings but just simple grep. I would not have predicted this
There have been many blogs like this over the last years. Yes, embeddings are computationally heavy, but they are not at all complicated and they provide a lot of benefit. 90% of "document" based RAG projects should view semantic search with embeddings as their primary method. It's very powerful and so easy to implement that you could try it out and discover whether performance would be an issue rather than trying to…
Embeddings are reasonably simple, but it’s a journey to get there, and I am very proud of the dog-heavy explainer I wrote on them: https://sgnt.ai/p/embeddings-explainer/
> we don’t especially want to say that books on forestry and similar to books on puppies
Here’s an even simpler take: just embed everything the first time, then track what was changed. Use a cheap model to summarize and clean up the documents/chats with summary and keywords. Unless you have entire libraries of books to embed it’s going to be a few hundred dollars of API calls. Then, throw it all in BigQuery. Handles all the vector stuff natively. Sprinkle an agentic bot UI thing on top to make it appear…
This assumes your text is small. Try embedding pdf reports - though luck. It surely won’t fit into most embeddings. I can think of many more examples: books, news articles, medical reports, insurance claims etc. they’re all too big to “index it all at once”
I worked on large scale RAG systems before and can say people vastly underestimate full text search and vastly overestimate embeddings. FTS is really easy, portable and scalable and gets you very far, the 80/20 rule applies. Embeddings appear to be nice and magic but when you really get into them you notice: semantic similarity isn’t as good as you think and certainly it won’t make everyone happy. You will inevitably…