141–150 of 247 posts
Earlier quoted context omitted.
Ask chatgpt next time. "What is rag in context of AI?"
Or just using a traditional search engine and "rag" plus literally any ML/AI/LLM term will yield a half dozen results at the top with "Retrieval-augmented generation" in the page title.
AWS Bedrock is fairly easy. You can do it in 5 or 6 clicks. You have to upload your documents to S3, create a “Knowledge Base” then sync your documents into a vector database like OpenSearch or PineCone. You are then good to go via their playground or the AWS API. I made a video here describing the process, check around 14 minutes in: https://ensembleanalytics.io/blog/introducing-bedrock-knowle... Bedrock is a decent…
I think the answer depends on how many documents you have. To think in terms of tokens (assuming 750-1000 tokens is a page), if you have a good estimate of number of pages you want to query on, you can decide on the approach. Three popular approaches: 1. RAG: Most popular and works really well on smaller datasets. It is limited by number of vectors/embeddings. A typical embedding could be of 1000 tokens in size. Llam…
> I talked to a firm with 70GB worth of data. No way a RAG pipeline would give them results. They are struggling to get LLMs to work for them. Wow so RAG is basically a toy for demos and low effort MVPs. 70GB is tiny, it’d barely qualify as “big data” 20 years ago. Is anyone trying more advanced stuff like knowledge graph augmented generation to try to expand on that?
LLM work best on use cases where the working context is the length of a short research paper (or less). Building with LLM is mostly a exercise in application engineering on how to get them the most relevant context at the right time and how to narrow its scope to produce reliable outputs.
Fine tuning can help specialize the LLM model to perform better, but AFAIK, the training sets are relatively small (in big data terms)
AWS Bedrock is fairly easy. You can do it in 5 or 6 clicks. You have to upload your documents to S3, create a “Knowledge Base” then sync your documents into a vector database like OpenSearch or PineCone. You are then good to go via their playground or the AWS API. I made a video here describing the process, check around 14 minutes in: https://ensembleanalytics.io/blog/introducing-bedrock-knowle... Bedrock is a decent…
Is there a limit? Could I create a knowledge base with 10,000 documents? 100k? 1M?
This isn't the same as the training data the LLM is trained on. As a result, it doesn't take advantage of the entire document set. So, if you have a billion documents, only 1-3 chunks will be picked for the final answer. When you know a question spans many many documents, the answer is never going to cover that.
You could make a recursive also where you parse all the chunks, generate summaries of those and then pass them to the next chunk sequentially and so on. But you can imagine how expensive and slow that will be. It might still work for you but this is a very lossy approach.
Earlier quoted context omitted.
Retrieval Augmented Generation - in brief, using some kind of search to find relevant documents to the user’s question (often vector DB search, which can search by “meaning”, by also other forms of more traditional search), then injecting those into the prompt to the LLM alongside the question, so it hopefully has facts to refer to (and its “generation” can be “augmented” by documents you’ve “retrieved”, I guess!)
So, as a contrived example, with RAG you make some queries, in some format, like “Who is Sauron?” And then start feeding in what books he’s mentioned in, paragraphs describing him from Tolkien books, things he has done. Then you start making more specific queries? How old is he, how tall is he, etc. And the game is you run a “questionnaire AI” that can look at a blob of text, and you ask it “what kind of questions mi…
You take the users question either embed it directly or augment it for embedding (you can for example use LLM to extract keywords form the question), query the vector db containing the data related to the question and then feed it all of LLM as: here is question form the user and here is some data that might be related to it.
AWS Bedrock is fairly easy. You can do it in 5 or 6 clicks. You have to upload your documents to S3, create a “Knowledge Base” then sync your documents into a vector database like OpenSearch or PineCone. You are then good to go via their playground or the AWS API. I made a video here describing the process, check around 14 minutes in: https://ensembleanalytics.io/blog/introducing-bedrock-knowle... Bedrock is a decent…
Bedrock is cool, but I found it prohibitively expensive for hobbyists and small companies. On a first glance, it would cost me something like $ 5,000 per month for a simple trained model.
I agree it is high and it is not exactly serverless. You pay by the hour.
Bedrock itself is charged based on tokens exchanged.
I think Pinecone is a cheaper database for hobby and small business projects, though I haven't looked into it.
How you do RAG with embeddings NOT in English? I mean there are a few thousand more languages.
You don't train on documents. There are many startups claiming that but they are deliberately using a misleading term because they know that's what people are searching for. You still do RAG. Llamaindex is still the best option that I know of. Most of the startups that have working products are likely using llamaindex. All of the ones that say they are training on documents are actually using RAG. Test it out. If it…
Not quite. It does work, albeit likely not optimal.