41–50 of 73 posts
Earlier quoted context omitted.
Could you explain or link to explanations of all of the acronyms you’ve used in your comment?
It makes me chuckle a bit to see this kind of request in a tech forum, particularly when discussing advanced LLM-related topics. This is akin to a HN comment asking someone to search the Internet for something on their behalf, while discussing search engine algorithms!
Earlier quoted context omitted.
It makes me chuckle a bit to see this kind of request in a tech forum, particularly when discussing advanced LLM-related topics. This is akin to a HN comment asking someone to search the Internet for something on their behalf, while discussing search engine algorithms!
It adds useful context to the discussion and spurs further conversation.
RAGAS: RAG Assessment [2]
RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval [3]
Self-RAG: Self-Reflective Retrieval-Augmented Generation [4]
Agentic RAG: Agentic Retrieval-Augmented Generation [5]
GraphRAG: Graph Retrieval-Augmented Generation [6]
[1] https://docs.haystack.deepset.ai/docs/hypothetical-document-...
[2] https://docs.ragas.io/en/stable/
[3] https://arxiv.org/html/2401.18059v1
[5] https://langchain-ai.github.io/langgraph/tutorials/rag/langg...
[6] https://www.microsoft.com/en-us/research/blog/graphrag-unloc...
To add some context, this isn't that novel of an approach. A common approach to improve RAG results is to "expand" the underlying chunks using an llm, so as to increase the semantic surface area to match against. You can further improve your results by running query expansion using HyDE[1], though it's not always an improvement. I use it as a fallback. I'm not sure what Anthropic is introducing here. I looked at the…
I was trying to do this using Prompt Caching like a month ago, but then noticed there's five minute maximum lifetime for the cached prompts - doesn't really work for my RAG needs (or probably most), where the queries would be ran during the next month or a year. I can't see any changes to that policy. Little surprised to see them talk about Prompt Caching relating to RAG.
Earlier quoted context omitted.
In short: no. The vector databases are here to store vectors and calculating distance between vectors. The embeddings model is the model that you pick to generate these vectors from a string or an image. You give "bart simpson" to an embeddings model and it becomes (43, -23, 2, 3, 4, 843, 34, 230, 324, 234, ...) You can imagine it like geometric points in space (well, it's a vector though), except that instead of bei…
that sucks :(. I wonder if there are other approaches to this, like simple word lookup, with storing a few synonyms, and prompting the LLM to always use the proper technical terms when performing a lookup.
My favorite thing about this is the way it takes advantage of prompt caching. That's priced at around 1/10th of what the prompts would normally cost if they weren't cached, which means that tricks like this (running every single chunk against a full copy of the original document) become feasible where previously they wouldn't have financially made sense. I bet there are all sorts of other neat tricks like this which…
You could do a lot of stuff with pre-calculating things for your embeddings. Why cache when you can pre-calculate. That brings into play a whole lot of things people commonly do as part of ETL. I come from a traditional search back ground. It's quite obvious to me that RAG is a bit of a naive strategy if you limit it to just using vector search with some off the shelf embedding model. Vector search simply isn't that…
The article claimed other context augmentation fails, and that you are better off paying anthropic to run an LLM on all your data, but it seems quite handwavy. What vector+text search nuance does a full document cache LLM rewrite catch that cheapo methods miss? Reminds me of "It is difficult to get a man to understand something when his salary depends on his not understanding it". (We process enough data that we try to limit LLMs to the retrieval step, and only embeddings & light LLMs to the indexing step, so it's a $$$ distinction for our customers.)
The context caching is neat in general, so I have to wonder if this use case is more about paying for ease than quality, and its value for quality is elsewhere.
An example: let's suppose you're using an LLM to play a multi user dungeon. In the past your character has behaved badly with taxis so that the game has decided to create a rule that says that whenever you try to enter a taxi you're kicked out: "we know who you are, we refuse to have you as a client until you formally apologize to the taxi company director". Upon apologizing, the rule is removed. Note that the director of the taxi company could be another player and be the one who issued the rule in the first place, to be enforced by his NPC fleet of taxis.
I'm wondering how well this could scale (with respect of number of active rules) and to which extent traditional RAG could be applied. It seems deciding whether a rule applies or not is a problem that is more abstract and difficult than deciding whether a chunk of knowledge is relevant or not.
In particular the main problem I have identified that makes it more difficult is the following dependency loop that doesn't appear with knowledge retrieval: you need to retrieve a rule to identify whether it applies or not. Does anyone know how this problem could be solved ?
I wonder how it would work if you generated the contexts yourself algorithmically. Depending on how well structured your docs are this could be quite trivial (eg for an html doc insert the title > h1 > h2 > chunk).
Waiting for the day when the entire AI industry goes back full circle to TF-IDF.
My favorite thing about this is the way it takes advantage of prompt caching. That's priced at around 1/10th of what the prompts would normally cost if they weren't cached, which means that tricks like this (running every single chunk against a full copy of the original document) become feasible where previously they wouldn't have financially made sense. I bet there are all sorts of other neat tricks like this which…