Baidu's Improving Retrieval Augmented Language Model with Self-Reasoning
1–6 of 6 posts
Re: Baidu's Improving Retrieval Augmented Language Model with Self-Reasoning
#2Can anyone explain what is gained by training a model? Why not use the foundational LLM for the relevance, evidence, and trajectory processes?
Re: Baidu's Improving Retrieval Augmented Language Model with Self-Reasoning
#3Can anyone explain what is gained by training a model? Why not use the foundational LLM for the relevance, evidence, and trajectory processes?
I assume you are referring to fine tuning a model here?
Re: Baidu's Improving Retrieval Augmented Language Model with Self-Reasoning
#4[deleted]
Re: Baidu's Improving Retrieval Augmented Language Model with Self-Reasoning
#5Can anyone explain what is gained by training a model? Why not use the foundational LLM for the relevance, evidence, and trajectory processes?
I assume you are referring to fine tuning a model here?
You could also just continue pre-training of an existing foundation model. Would still be cheaper by not starting from zero.
Re: Baidu's Improving Retrieval Augmented Language Model with Self-Reasoning
#6Earlier quoted context omitted.
I assume you are referring to fine tuning a model here?
You could also just continue pre-training of an existing foundation model. Would still be cheaper by not starting from zero.
The amount of accuracy while doing fine tuning or distillation is usually better than pre-training an existing model, not to mention the graph against the cost.