If you are using LLM RAG – you should be doing RAFT
techcommunity.microsoft.com
If you are using LLM RAG – you should be doing RAFT
1–10 of 15 posts
Re: If you are using LLM RAG – you should be doing RAFT
#2This is a three-way collaboration between Berkeley AI, Microsoft Azure, and Meta AI!
RAFT involves the concept of domain-specific RAG, which represents a more focused and growingly favored area compared to the broader concept of the general open-book exam. In such exams, the domain in which the LLM will be evaluated is known in advance and used for inference. The LLM is capable of addressing prompts by leveraging any and all information from this particular domain, on which it has been specifically fine-tuned.
Re: If you are using LLM RAG – you should be doing RAFT
#3It's "Retrieval Augmented Fine Tuning". The related blog post is interesting: https://gorilla.cs.berkeley.edu/blogs/9_raft.html
Re: If you are using LLM RAG – you should be doing RAFT
#4Page me when this process is at least partially automated and continuous.
Re: If you are using LLM RAG – you should be doing RAFT
#5Does any of this matter when you have 5 million token input and can just shove everything into the input?
Re: If you are using LLM RAG – you should be doing RAFT
#6Does any of this matter when you have 5 million token input and can just shove everything into the input?
Yes, because 5 million token input is slow, expensive, and error-prone.
Re: If you are using LLM RAG – you should be doing RAFT
#7> They hypothesized that a student who studies the textbooks before the open-book exam was likely to perform better than a student who studies the textbook.
interesting hypothesis, but probably not what was meant.
“editor” is shaping up to be the hot new job for fleshy consciousnesses by the early thirties.
Re: If you are using LLM RAG – you should be doing RAFT
#8Am I missing the point or is this not how everyone was doing RAG before?
I've had my much greater success doing the RAG process on a fine tuned model.
Re: If you are using LLM RAG – you should be doing RAFT
#9Am I missing the point or is this not how everyone was doing RAG before? I've had my much greater success doing the RAG process on a fine tuned model.
> Am I missing the point or is this not how everyone was doing RAG before?
No, especially people wrapping RAG frameworks around models they don’t have access to fine tune (e.g., GPT-4 if you aren't OpenAI and/or Microsoft.)
Edit: Of course, further evidence that this method is useful doesn't help people in that condition, though.
Re: If you are using LLM RAG – you should be doing RAFT
#10Does any of this matter when you have 5 million token input and can just shove everything into the input?
How to learn quadratic scaling the hard way.