11–20 of 26 posts
What are other than fine-tuning methods to make LLM smarter? Im familair with RAG - Retrival Augumented Generation.
RLHF is a popular candidate, but the focus is more on "helpfulness" and "safety" -- I don't think it necessarily improves LLMs on reasoning benchmarks
Earlier quoted context omitted.
RLHF is a popular candidate, but the focus is more on "helpfulness" and "safety" -- I don't think it necessarily improves LLMs on reasoning benchmarks
if anything, RLHF makes the model dumber, not smarter.
What are other than fine-tuning methods to make LLM smarter? Im familair with RAG - Retrival Augumented Generation.
What are other than fine-tuning methods to make LLM smarter? Im familair with RAG - Retrival Augumented Generation.
Here's one library to do this https://github.com/guidance-ai/guidance
I have wondered if the very big models trained on a Big Pile of Everything can be used to curate smaller, higher quality data sets that lead to high performing models with smaller parameter counts. Not only are smaller models easier to distribute and faster at inference time, but it offers a licensing escape hatch if future copyright law changes or court rulings make it hard to publicly offer models trained on non-pe…
Not sure on the licensing but yes you can do that technically. Phi-1 and therefore phi-1.5 are partially trained on gpt3.5 generated synthetic textbooks.
I have wondered if the very big models trained on a Big Pile of Everything can be used to curate smaller, higher quality data sets that lead to high performing models with smaller parameter counts. Not only are smaller models easier to distribute and faster at inference time, but it offers a licensing escape hatch if future copyright law changes or court rulings make it hard to publicly offer models trained on non-pe…
I have wondered if the very big models trained on a Big Pile of Everything can be used to curate smaller, higher quality data sets that lead to high performing models with smaller parameter counts. Not only are smaller models easier to distribute and faster at inference time, but it offers a licensing escape hatch if future copyright law changes or court rulings make it hard to publicly offer models trained on non-pe…
Doesn't that lead to model collapse?
I was hoping that this would go more into the details of dataset selection and what makes for high-quality data, but it seems to be more a prelude to a Lit-GPT advertisement :/
I need to train more models to see if this is an accurate claim, but I've been finishing up the storage layer and haven't gotten to that yet.
What are other than fine-tuning methods to make LLM smarter? Im familair with RAG - Retrival Augumented Generation.
Keyterms can be used in the prompt to drive the LLM to better grounded responses as well as helping locate relevant embeddings for RAG, when vector search isn't enough.
Another consideration is writing code for processing things that look similar. For example, one might have the LLM write regex code which is then tested to work and put into production in a pipeline to parse log files, or write SQL off conversational queries, which are then run against a database.
What would a good fine-tuning dataset for language translation look like? I want to try fine-tuning to machine translate to and from a fairly niche language ( https://en.wikipedia.org/wiki/S'gaw_Karen_language) . How much text would I need, and what format would be ideal? I have a number of book length texts, most only in the target language, and a few bilingual or multilingual. For the bilingual and multilingual tex…
As noted below, extracting words or keyterms would maybe be a good idea, as they could be included in the training set.
The training set would the be comprised of the prompt, the translation, and keyterms. As you will want to vet the generated texts anyway, you could then decide if the foundational model was working enough. You could also try to run the largest "open" model you could find on the prompts, to see if those needed training as well. There are many different Llama models trained on HuggingFace for language pairs, so see if your languages are already built and test those.
I'm building a simple, Open Source ML pipeline manager at https://ai.featurebase.com/. I'd be down to help you with this!